Systems and methods for a 3d home model for visualizing proposed changes to home

ABSTRACT

The following relates generally to light detection and ranging (LIDAR) and artificial intelligence (AI). In some embodiments, a system: receives LIDAR data generated from a LIDAR camera; measures a plurality of dimensions of a room of the home based upon processor analysis of the LIDAR data; builds a 3D model of the room based upon the measured plurality of dimensions; receives an indication of a proposed change to the room; modifies the 3D model to include the proposed change to the room; and displays a representation of the modified 3D model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/240,970, (filed (Apr. 26, 2021), which claims the benefit of U.S.Provisional Application No. 63/016,168 (filed Apr. 27, 2020); U.S.Provisional Application No. 63/025,600 (filed May 15, 2020); and U.S.Provisional Application No. 63/027,201 (filed May 19, 2020), theentirety of each of which is incorporated by reference herein.

FIELD

The present disclosure generally relates to light detection and ranging(LIDAR) technology and artificial intelligence (AI). More specifically,the following relates to LIDAR technology and AI based 3-dimensional(3D) models, navigation systems, and visualization systems.

BACKGROUND

LIDAR is a technology that measures distance to a target by illuminatingthe target (e.g., using laser light) and then measuring the reflectedlight with a sensor (e.g., measuring the time of flight from the lasersignal source to its return to the sensor). Digital 3D representationsof the target may then be made using differences in laser return timesand wavelengths. LIDAR may be used to measure distances (e.g., thedistance from a LIDAR camera to an object, the distance between objects,and so forth).

SUMMARY

The present embodiments may be related to LIDAR technology, and to AI.Broadly speaking, some embodiments relate to: (i) LIDAR technology based3D home models for visualizing proposed changes to a home; (ii) LIDARtechnology based 3D home models for representation of the home; (iii)LIDAR technology based viewing of objects to be placed in a building;(iv) AI based recommendations for placement of belongings in aresidence; (v) LIDAR technology based visualization of landscape design;(vi) LIDAR technology based visualization of utility lines; (vii) LIDARtechnology based commercial inventory mapping; (viii) LIDAR technologyand AI based floor plan generation; and (ix) LIDAR technology and AIbased visualization of directions to interior rooms.

In accordance with the described embodiments, the disclosure hereingenerally addresses, inter alia, systems and methods for visualizingproposed changes to a home. A server may receive light detection andranging (LIDAR) data generated from a LIDAR camera, measure a pluralityof dimensions of a room of the home based upon processor analysis of theLIDAR data, build a 3D model of the room based upon the measuredplurality of dimensions, receive an indication of a proposed change tothe room, modify the 3D model to include the proposed change to theroom, and display a representation of the modified 3D model.

In one aspect, a computer-implemented method for visualizing proposedchanges to a home may be provided. The computer-implemented method mayinclude, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; (2) measuring a plurality ofdimensions of the home based upon processor analysis of the LIDAR data;(3) building a 3D model of the home based upon the measured plurality ofdimensions; (4) receiving an indication of a proposed change to the; (5)modifying the 3D model to include the proposed change to the room;and/or (6) displaying a representation of the modified 3D model. Themethod may include additional, less, or alternate actions, includingthat discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forrepresentation of property. A server may receive light detection andranging (LIDAR) data generated from a LIDAR camera, measure plurality ofdimensions of the home based upon processor analysis of the LIDAR data,build a 3D model of the home based upon the measured plurality ofdimensions, and display a representation of the 3D model by visuallynavigating through the 3D model.

In another aspect, a computer-implemented method for representation ofproperty may be provided. The computer-implemented method may include,via one or more local or remote processors, transceivers, sensors,and/or servers, (1) receiving light detection and ranging (LIDAR) datagenerated from a LIDAR camera; (2) measuring a plurality of dimensionsof the home based upon processor analysis of the LIDAR data; (3)building a 3D model of the home based upon the measured plurality ofdimensions; and/or (4) displaying a representation of the 3D model byvisually navigating through the 3D model. The method may includeadditional, less, or alternate actions, including that discussedelsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods for viewingpotential placement of an object. A server may receive light detectionand ranging (LIDAR) data generated from a LIDAR camera, measure aplurality of dimensions of the object based upon processor analysis ofthe LIDAR data, receive or generate a 3D model of a room, the 3D modelof the room including dimensional data of the room, insert arepresentation of the object into the 3D model of the room based uponprocessor analysis of: (i) the plurality of dimensions of the objectmeasured from the LIDAR data; and (ii) the dimensional data of the room,and display the 3D model of the room with the inserted representation ofthe object.

In another aspect, a computer-implemented method for viewing potentialplacement of an object may be provided. The computer-implemented methodmay include, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; (2) measuring a plurality ofdimensions of the object based upon processor analysis of the LIDARdata; (3) receiving or generating a 3D model of a room, the 3D model ofthe room including dimensional data of the room; (4) inserting arepresentation of the object into the 3D model of the room based uponprocessor analysis of: (i) the plurality of dimensions of the objectmeasured from the LIDAR data; and (ii) the dimensional data of the room;and/or (5) displaying the 3D model of the room with the insertedrepresentation of the object. The method may include additional, less,or alternate actions, including that discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods for machinelearning based recommendation of object placement. A server may train amachine learning algorithm based upon preexisting data of objectplacement in a room, receive room data comprising dimensional data of aroom, receive object data comprising: (i) dimensional data of an object;(ii) a type of the object; and/or (iii) color data of the object, andwith the trained machine learning algorithm, generate a recommendationfor placement of the object in the room based upon: (i) the receivedroom data, and (ii) the received object data.

In another aspect, a computer-implemented method for machine learningbased recommendation of object placement may be provided. Thecomputer-implemented method may include, via one or more local or remoteprocessors, transceivers, sensors, and/or servers, (1) training amachine learning algorithm based upon preexisting data of objectplacement in a room; (2) receiving room data comprising dimensional dataof a room; (3) receiving object data comprising: (i) dimensional data ofan object; (ii) a type of the object; and/or (iii) color data of theobject; (4) and/or with the trained machine learning algorithm,generating a recommendation for placement of the object in the roombased upon: (i) the received room data, and (ii) the received objectdata. The method may include additional, less, or alternate actions,including that discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forvisualization of landscape design. A server may receive light detectionand ranging (LIDAR) data generated from a LIDAR camera, measure aplurality of dimensions of a landscape based upon processor analysis ofthe LIDAR data, build a 3D model of the landscape based upon themeasured plurality of dimensions, the 3D model including: (i) astructure, and (ii) a vegetation, and display a representation of the 3Dmodel.

In another aspect, a computer-implemented method for visualization oflandscape design may be provided. The computer-implemented method mayinclude, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; (2) measuring a plurality ofdimensions of a landscape based upon processor analysis of the LIDARdata; (3) building a 3D model of the landscape based upon the measuredplurality of dimensions, the 3D model including: (i) a structure, and(ii) a vegetation; and/or (4) displaying a representation of the 3Dmodel. The method may include additional, less, or alternate actions,including that discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forvisualization of a utility line. A server may receive light detectionand ranging (LIDAR) data generated from a LIDAR camera, receivepreexisting utility line data, and determine a location of the utilityline based upon: (i) the received LIDAR data, and (ii) the receivedpreexisting utility line data.

In another aspect, a computer-implemented method for visualization of autility line may be provided. The computer-implemented method mayinclude, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; (2) receiving preexistingutility line data; and/or (3) determining a location of the utility linebased upon: (i) the received LIDAR data, and (ii) the receivedpreexisting utility line data. The method may include additional, less,or alternate actions, including that discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forcommercial inventory mapping. A server may receive light detection andranging (LIDAR) data generated from a LIDAR camera, determine data of afirst object based upon processor analysis of the LIDAR data, the dataof the first object comprising: (i) dimensional data of the firstobject, and (ii) a type of the first object, and add the first objectand the first object data to a commercial inventory list.

In another aspect, a computer-implemented method for commercialinventory mapping may be provided. The computer-implemented method mayinclude, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; (2) determining data of afirst object based upon processor analysis of the LIDAR data, the dataof the first object comprising: (i) dimensional data of the firstobject, and (ii) a type of the first object; and/or (3) adding the firstobject and the first object data to a commercial inventory list. Themethod may include additional, less, or alternate actions, includingthat discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forcommercial inventory mapping including a LIDAR-based virtual map. Aserver may receive light detection and ranging (LIDAR) data generatedfrom a LIDAR camera via wireless communication or data transmission overone or more radio frequency links, the LIDAR data associated with astore or a store layout; generate a LIDAR-based virtual map of the storefrom processor analysis of the LIDAR data; determine locations ofindividual goods in the store; overlay the locations of the individualgoods onto the LIDAR-based virtual map; and generate an updatedLIDAR-based virtual map of the store displaying aisles of the store andthe overlaid locations of the individual goods within the store.

In another aspect, a computer-implemented method for commercialinventory mapping including a LIDAR-based virtual map may be provided.The computer-implemented method may include, via one or more local orremote processors, transceivers, sensors, and/or servers, (1) receivingLIDAR data generated from a LIDAR camera, the LIDAR data associated witha store or a store layout; (2) generating a LIDAR-based virtual map ofthe store from processor analysis of the LIDAR data; (3) determininglocations of individual goods in the store; (4) overlaying the locationsof the individual goods onto the LIDAR-based virtual map; and (5)generating an updated LIDAR-based virtual map of the store displayingaisles of the store and the overlaid locations of the individual goodswithin the store.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods forcommercial inventory mapping including determining if goods are stillavailable. A server may receive sensor data via wireless communicationor data transmission over one or more radio frequency links, the sensordata associated with item movement or purchase, the sensor data beinggenerated from a good-mounted sensor, shelf-mounted sensor, a camera, ora self-check device; update an electronic inventory of goods within astore based upon the received sensor data associated with the itemmovement or purchase; receive an electronic order of goods from acustomer mobile device via wireless communication or data transmissionover one or more radio frequency links; determine if the goods in theelectronic order received from the customer are still available bycomparing the updated electronic inventory of goods with the electronicorder of goods and/or comparing the electronic order of goods with otherincoming electronic orders from other customers; generate a LIDAR-basedvirtual map of the store from processor analysis of LIDAR data;determine a location of the goods in the electronic order that are stillavailable; overlay the determined location of the goods onto theLIDAR-based virtual map of the store; and generate and display anupdated LIDAR-based virtual map of the store displaying aisles of thestore and the determined location of the goods within the store.

In another aspect, a computer-implemented method for commercialinventory mapping including determining if goods are still available maybe provided. The computer-implemented method may include, via one ormore local or remote processors, sensors, servers, light detection andranging (LIDAR) devices, and/or transceivers, (1) receiving sensor datavia wireless communication or data transmission over one or more radiofrequency links, the sensor data associated with item movement orpurchase, the sensor data being generated from a good-mounted sensor,shelf-mounted sensor, a camera, or a self-check device; (2) updating anelectronic inventory of goods within a store based upon the receivedsensor data associated with the item movement or purchase; (3) receivingan electronic order of goods from a customer mobile device via wirelesscommunication or data transmission over one or more radio frequencylinks; (4) determining goods in the electronic order received from thecustomer that are still available by comparing the updated electronicinventory of goods with the electronic order of goods and/or comparingthe electronic order of goods with other incoming electronic orders fromother customers; (5) generating a LIDAR-based virtual map of the storefrom processor analysis of LIDAR data; (6) determining a location of thegoods in the electronic order that are still available; (7) overlayingthe determined location of the goods onto the LIDAR-based virtual map ofthe store; and (8) generating an updated LIDAR-based virtual map of thestore displaying aisles of the store and the determined locations of thegoods within the store.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods for 3Dgeneration of a floor plan for a commercial building. A server mayreceive a 3-dimensional (3D) model of a floor of a commercial buildingcomprising a plurality of dimensions of the floor of the commercialbuilding, and with a machine learning algorithm, generate a new floorplan of the floor of the commercial building based upon the received 3Dmodel of the floor. The generated new floor plan may be a 3D floor plan.

In another aspect, a computer-implemented method for 3D generation of afloor plan for a commercial building may be provided. Thecomputer-implemented method may include, via one or more local or remoteprocessors, transceivers, sensors, and/or servers, (1) receiving a3-dimensional (3D) model of a floor of a commercial building comprisinga plurality of dimensions of the floor of the commercial building;and/or (2) with a machine learning algorithm, generating a new floorplan of the floor of the commercial building based upon the received 3Dmodel of the floor; wherein the generated new floor plan comprises a 3Dfloor plan. The method may include additional, less, or alternateactions, including that discussed elsewhere herein.

Further in accordance with the described embodiments, the disclosureherein generally addresses, inter alia, systems and methods for 3Dnavigation of an interior of a building. A server may receive a3-dimensional (3D) model of the building, the 3D model comprising: (i) aplurality of dimensions of the interior of the building, and (ii) alocation of a room and/or a location of a commercial item, receive, froma user, a request for navigation instructions to the room and/or thecommercial item, calculate the navigation instructions based upon thereceived 3D model of the building, and provide, to the user, thecalculated navigation instructions to the room and/or the commercialitem.

In another aspect, a computer-implemented method for 3D navigation of aninterior of a building may be provided. The computer-implemented methodmay include, via one or more local or remote processors, transceivers,sensors, and/or servers, (1) receiving a 3-dimensional (3D) model of thebuilding, the 3D model comprising: (i) a plurality of dimensions of theinterior of the building, and (ii) a location of a room and/or alocation of a commercial item; (2) receiving, from a user, a request fornavigation instructions to the room and/or the commercial item; (3)calculating the navigation instructions based upon the received 3D modelof the building; and/or (4) providing, to the user, the calculatednavigation instructions to the room and/or the commercial item. Themethod may include additional, less, or alternate actions, includingthat discussed elsewhere herein.

Advantages will become apparent to those skilled in the art from thefollowing description. For example, in one aspect, the systems andmethods disclosed herein advantageously produce a more accurate (i) 3Dmodels, (ii) navigation systems, and (iii) visualization systems thanprior systems. This is, in part, because of the higher accuracy thatLIDAR technology provides in measuring dimensions, such as dimensions ofobjects, rooms, hallways, etc. In another aspect, a further advantage ofthe systems and methods described herein is to provide a company with amore accurate inventory, including specific locations of items. In yetanother aspect, a further advantage of the systems and methods describedherein is to allow a customer of a store to minimize time spent in thestore. Further advantages will become apparent to those of ordinaryskill in the art from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Advantages will become more apparent to those skilled in the art fromthe following description of the preferred embodiments which have beenshown and described by way of illustration. As will be realized, thepresent embodiments may be capable of other and different embodiments,and their details are capable of modification in various respects.Accordingly, the drawings and description are to be regarded asillustrative in nature and not as restrictive.

The Figures described below depict various aspects of the applications,methods, and systems disclosed herein. It should be understood that eachFigure depicts an embodiment of a particular aspect of the disclosedapplications, systems and methods, and that each of the Figures isintended to accord with a possible embodiment thereof. Furthermore,wherever possible, the following description refers to the referencenumerals included in the following Figures, in which features depictedin multiple Figures are designated with consistent reference numerals.

FIG. 1 shows an exemplary computer system for generating a 3D home modelfor visualizing proposed changes to the home.

FIG. 2 shows a flowchart of an exemplary computer-implemented method ofgenerating a 3D home model for visualizing proposed changes to home.

FIG. 3 shows an exemplary computer system for representation of a home.

FIG. 4A shows a flowchart of an exemplary computer-implemented methodfor representation of a home, including navigating through a 3D modelaccording to an input from a human user. FIG. 4B shows a flowchart of anexemplary computer-implemented method for representation of a home,including navigating through a 3D model according to a defaultnavigation generated by a machine learning program.

FIG. 5 shows an exemplary computer system for viewing potentialplacement of an object in a room.

FIG. 6 shows a flowchart of an exemplary computer-implemented method forviewing potential placement of an object in a room.

FIG. 7 shows an exemplary computer system for AI based recommendationsfor object placement in a home.

FIG. 8 shows a flowchart of an exemplary computer-implemented method forAI based recommendations for object placement in a home.

FIG. 9 shows an exemplary computer system for visualization of landscapedesign.

FIG. 10 shows a flowchart of an exemplary computer-implemented methodfor visualization of landscape design.

FIG. 11 shows an exemplary computer system for visualization of autility line.

FIG. 12 shows a flowchart of an exemplary computer-implemented methodfor visualization of a utility line.

FIG. 13 shows an exemplary computer system for commercial inventorymapping.

FIG. 14A shows a flowchart of an exemplary computer-implemented methodfor commercial inventory mapping.

FIG. 14B shows a flowchart of another exemplary computer-implementedmethod for commercial inventory mapping.

FIG. 14C shows a flowchart of another exemplary computer-implementedmethod for commercial inventory mapping.

FIG. 15 shows an exemplary computer system for 3D generation of a floorplan for a commercial building.

FIG. 16 shows a flowchart of an exemplary computer-implemented method 3Dgeneration of a floor plan for a commercial building.

FIG. 17 shows an exemplary computer system for 3D navigation of aninterior of a building.

FIG. 18 shows a flowchart of an exemplary computer-implemented methodfor 3D navigation of an interior of a building.

DETAILED DESCRIPTION

The present embodiments relate to, inter alia: (i) LIDAR technologybased 3D home models for visualizing proposed changes to a home; (ii)LIDAR technology based 3D home models for representation of the home;(iii) LIDAR technology based viewing of objects to be placed in abuilding; (iv) artificial intelligence (AI) based recommendations forplacement of belongings in a residence; (v) LIDAR technology basedvisualization of landscape design; (vi) LIDAR technology basedvisualization of utility lines; (vii) LIDAR technology based commercialinventory mapping; (viii) LIDAR technology and AI based floor plangeneration; and (ix) LIDAR technology and AI based visualization ofdirections to interior rooms.

LIDAR is a technology that measures distance to a target by illuminatingthe target (e.g., using laser light) and then measuring the reflectedlight with a sensor (e.g., measuring the time of flight from the lasersignal source to its return to the sensor). Digital 3D representationsof the target can then be made using differences in laser return timesand wavelengths. LIDAR may be used to measure distances (e.g., thedistance from a LIDAR camera to an object, the distance between objects,and so forth). Moreover, LIDAR is able to make more accuratemeasurements of dimensions of objects than previously knowntechnologies.

In this respect, LIDAR may create a 3D point cloud model (e.g., a set ofdata points in space) of a room or landscape by measuring many points inthe room or landscape. Furthermore, as is understood in the art, 3Dpoint clouds may be converted to 3D surfaces (e.g., by using techniquessuch as Delaunay triangulation, alpha shapes, or ball pivoting to builda network of triangles over existing vertices of the point cloud).

Exemplary System for a 3D Home Model for Visualizing Proposed Changes toHome

FIG. 1 shows an exemplary computer system for generating a 3D home modelfor visualizing proposed changes to the home. With reference thereto,servers 110 are shown sending and receiving information with LIDARcamera 120. The gathered LIDAR data may be analyzed to determine thedimensions of a house, including dimensions of part or all of theinterior and exterior of the house, as well as specific dimensions ofany part of a room (e.g., the length, width, and/or height of a wall, orof an object in the room, etc.). The LIDAR data may be used to create apartial or complete home model, and/or a partial or complete room model.Furthermore, 3D point cloud(s) may be created from the LIDAR data. TheLIDAR camera 120 may be operated by any human or machine.

In some embodiments, the LIDAR camera 120 may be operated by thehomeowner. For instance, a home owner may be planning on remodeling akitchen. In accordance with the techniques described herein, thehomeowner may operate LIDAR camera 120 to generate data of the kitchen,which may be used to create a 3D model of the kitchen and further usedto visualize proposed changes to the kitchen.

In some embodiments, the LIDAR camera 120 may be operated by a companyemployee. For instance, an employee of a kitchen remodeling company maybring a LIDAR camera to an individual's house, and use the LIDAR camera120 to gather LIDAR data from the kitchen. The LIDAR data may then beused to create a 3D model of the kitchen, and visualize proposed changesto the kitchen.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the home. For instance, data may be gathered frompublic records, property deeds, government records, realtors (e.g., fromwebsites and apps that realtors post information to), websites (e.g.,websites that display information of houses for sale), previousinsurance claims, and so forth. In some embodiments, this data isgathered from preexisting home database 130. The database 130 mayinclude structural data of the home.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Each server 110 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system for a 3D home model for visualizing proposed changes 100, inaddition to other software applications. The server 110 may furtherinclude a database 146, which may be adapted to store data related tothe LIDAR camera 120, as well as any other data. The server 110 mayaccess data stored in the database 146 when executing various functionsand tasks associated with LIDAR technology and generating 3D home modelsalong with proposed changes to the home.

Although the exemplary system 100 is illustrated to include one LIDARcamera 120, one drone 140, database 130, photographic camera 125 and onegroup of servers 110 (FIG. 1 is illustrated to show three servers 110,but it should be understood that the server(s) 110 may be one or moreserver(s)), it should be understood that different numbers of LIDARcamera 120, drone 140, database 130, photographic camera 125, and/orservers 110 may be utilized. For instance, the system 100 may includeany number of servers 110 and hundreds of mobile LIDAR cameras 120 ordrones 140. Furthermore, the database storage or processing performed bythe one or more servers 110 may be distributed among a plurality ofservers 110 in an arrangement known as “cloud computing.” Thisconfiguration may provide various advantages, such as enabling nearreal-time uploads and downloads of information as well as periodicuploads and downloads of information.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162. Similarly, thememory of the controller 155 may include multiple RAMs 164 and multipleprogram memories 160. Although the I/O circuit 166 is shown as a singleblock, it should be appreciated that the I/O circuit 166 may include anumber of different types of I/O circuits. The RAM 164 and programmemories 160 may be implemented as semiconductor memories, magneticallyreadable memories, or optically readable memories, for instance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a receiving preexisting housedata application 143 for receiving preexisting data; camera datareceiving application 144 for receiving camera data; a 3D modelbuilder/modifier 145 for building and modifying a 3D model (e.g., of ahome, room, object etc.); a display application 146 for displaying arepresentation of the 3D model; and a navigation input receiver 147 forreceiving navigation input. The various software applications may beexecuted on the same computer processor or on different computerprocessors.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110. The machine learning algorithm may be asupervised learning algorithm, employ decision trees, make use of anartificial neural network, make use of Bayesian statistical analysis, orcombinations thereof. In this regard, a processor or a processingelement may be trained using supervised or unsupervised machinelearning, and the machine learning algorithm may employ a neuralnetwork, which may be a convolutional neural network, a deep learningneural network, or a combined learning module or program that learns intwo or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

It should be understood that, over time, the servers 110 may accumulatea large pool of data on an individual home or a group of homes.

The data described above may be used (e.g., with a machine learningalgorithm described above or by any other technique) to generate a 3Dhome model for visualizing proposed changes to the home. The machinelearning algorithm may be trained using previously known home data.

Exemplary Computer-Implemented Method of Generating a 3D Home Model forVisualizing Proposed Changes to Home

FIG. 2 shows a flowchart of an exemplary computer-implemented method 200of generating a 3D home model for visualizing proposed changes to home.With reference thereto, at step 202, LIDAR data (e.g., from LIDAR camera120) is received, such as via wireless communication or datatransmission over one or more radio frequency links, by server 110. Insome embodiments, the LIDAR data is transferred to the servers 110 fromthe LIDAR camera 120 via a physical memory device.

At optional step 204, the server 110 receives any of: camera data fromcamera 125; preexisting home data from database 130; and/or drone datafrom drone 140. At step 206, the servers 110 measure a plurality ofdimensions of a room of the home based upon processor analysis of theLIDAR data. In this respect, LIDAR data provides more accuratemeasurements than other measurement methods (e.g., measurement data fromphotographic camera data alone). The dimensions measured may includedimensions of any object and/or wall of a room.

At step 208, the servers 110 build a 3D model of the house or part ofthe house (e.g., one or more rooms, etc.) based upon the deriveddimensions. The 3D model may further be based upon camera data fromcamera 125; preexisting home data from database 130; and/or drone datafrom drone 140. For instance, the model may include dimensional data ofan object from the LIDAR data, and color data of the object from thecamera data. In another aspect, structural data from database 130 mayshow where a beam needed for support is, and this may be depicted in the3D model. For instance, the 3D model may show the wall with a cut outindicating where the necessary support beam; or, the necessary supportbeam may be indicated by a different color that contrasts it from thewall covering the necessary support beam. In another aspect, drone data(e.g., a LIDAR camera, a photographic camera, or any other data comingfrom a drone inside or outside of the building) may aide in building the3D model. For instance, a drone flying exterior to the house may providedata verifying where a window or skylight is.

At step 210, the servers 110 receive an indication of a proposed changeor repair to the room (e.g., addition or removal of a wall or skylight;changes of a kitchen or bathroom remodeling project; repair to a windowor wall; etc.). For instance, in a kitchen remodel, the proposed changemay suggest a different size or color of a countertop or cabinet. In abathroom remodel, the proposed change may be, for instance, a differentfloor and/or wall tiles; a different shower or bathtub; a differentmirror; etc. The servers 110 may also provide an estimated cost to makethe change (e.g., an estimated cost of different cabinets or countertopsviewed in the 3D model).

In another aspect the proposed change may be a proposed change for arepair. For instance, following a fire stemming from an electricaloutlet, only part of the wall may be damaged (e.g., the part around theelectrical outlet); here, the proposed change/repair may be to replaceonly part of the wall around the electrical outlet. Furthermore, theservers 110 may provide a cost estimate of making the repair (e.g., inthe electrical outlet fire example, the cost would depend on how much ofthe wall was damaged).

At step 212, the 3D model is modified to include the proposed change tothe room. At step 214, a representation of the modified 3D model isdisplayed. This may optionally comprise displaying a 2D image from themodel. In some embodiments, the user may navigate through the 3D modelsusing arrows (e.g., the user supplies navigational data to the servers110 by clicking on the arrows of the 3D model).

Exemplary System for Generating a 3D Model for Representing a Home

FIG. 3 shows an exemplary computer system for generating a 3D home modelfor representing a home. With reference thereto, servers 110 are shownsending and receiving information with LIDAR camera 120. The gatheredLIDAR data may be analyzed to determine the dimensions of a house,including dimensions of part or all of the interior and exterior of thehouse, as well as specific dimensions of any part of a room (e.g., thelength, width, and/or height of a wall, or of an object in the room,etc.). The LIDAR data may be used to create a partial or complete homemodel, and may include 3D point cloud(s) created from the LIDAR data.The LIDAR camera 120 may be operated by any human or machine.

In some embodiments, the LIDAR camera 120 may be operated by thehomeowner. For instance, a home owner may be planning to move, and thusmay wish to market the home on online websites. In accordance with thetechniques described herein, the homeowner may operate LIDAR camera 120to generate data of the home, which may be used to create a 3D model ofthe home for online display.

In some embodiments, the LIDAR camera 120 may be operated by a realestate agent or a company employee (e.g., an employee of an onlinewebsite for marketing houses). For instance, a real estate agent or acompany employee may bring a LIDAR camera to an individual's house, anduse the LIDAR camera 120 to gather LIDAR data from the home. The LIDARdata may then be used to create a 3D model of the home, and display the3D model on an online website.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent through theinternet, sent by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the home. For instance, data may be gathered frompublic records, property deeds, government records, realtors (e.g., fromwebsites and apps that realtors post information to), websites (e.g.,websites that display information of houses for sale), previousinsurance claims, and so forth. In some embodiments, this data isgathered from preexisting home database 130. The database 130 may alsoinclude structural data of the home.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Each server 110 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system for a 3D home model for representation of a home 300, inaddition to other software applications. The server 110 may furtherinclude a database 146, which may be adapted to store data related tothe LIDAR camera 120, as well as any other data. The server 110 mayaccess data stored in the database 146 when executing various functionsand tasks associated with LIDAR technology and generating 3D home modelsalong with proposed changes to the home.

Although the exemplary system 300 is illustrated to include one LIDARcamera 120, one drone 140, database 130, photographic camera 125, andone group of servers 110 (FIG. 3 is illustrated to show three servers110, but it should be understood that the server(s) 110 may be one ormore server(s)), it should be understood that different numbers of LIDARcamera 120, drone 140, database 130, photographic camera 125, and/orservers 110 may be utilized. For instance, the system 300 may includeany number of servers 110 and hundreds of mobile LIDAR cameras 120 ordrones 140. Furthermore, the database storage or processing performed bythe one or more servers 110 may be distributed among a plurality ofservers 110 in an arrangement known as “cloud computing.” Thisconfiguration may provide various advantages, such as enabling nearreal-time uploads and downloads of information as well as periodicuploads and downloads of information.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165.

It should be appreciated that although only one microprocessor 162 isshown, the controller 155 may include multiple microprocessors 162.Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a receiving preexisting housedata application 143 for receiving preexisting data; camera datareceiving application 144 for receiving camera data; a 3D modelbuilder/modifier 145 for building and modifying a 3D model (e.g., of ahome, room, object etc.); a display application 146 for displaying arepresentation of the 3D model; a navigation input receiver 147 forreceiving navigation input; and/or default navigation calculator 302 forcalculating a default navigation through the 3D model. The varioussoftware applications may be executed on the same computer processor oron different computer processors.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110. The machine learning algorithm may be asupervised learning algorithm, employ decision trees, make use of anartificial neural network, make use of Bayesian statistical analysis, orcombinations thereof. In this regard, a processor or a processingelement may be trained using supervised or unsupervised machinelearning, and the machine learning algorithm may employ a neuralnetwork, which may be a convolutional neural network, a deep learningneural network, or a combined learning module or program that learns intwo or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

It should be understood that, over time, the servers 110 may accumulatea large pool of data on an individual home or a group of homes.

The data described above may be used (e.g., with a machine learningalgorithm described above or by any other technique) to generate a 3Dhome model for representing the home. The machine learning algorithm maybe trained using previously known data.

Exemplary Computer-Implemented Method of Generating a 3D Model forRepresenting a Home

FIG. 4A shows a flowchart of an exemplary computer-implemented method400A for representation of a home, including navigating through a 3Dmodel according to an input from a human user. With reference thereto,at step 404, LIDAR data (e.g., from LIDAR camera 120) is received, suchas via wireless communication or data transmission over one or moreradio frequency links, by server 110. In some embodiments, the LIDARdata is transferred to the servers 110 from the LIDAR camera 120 via aphysical memory device.

At optional step 404, the server 110 receives any of: camera data fromcamera 125; preexisting home data from database 130; and/or drone datafrom drone 140. At step 406, the servers 110 measure a plurality ofdimensions of the home based upon processor analysis of the LIDAR data.In this respect, LIDAR data provides more accurate measurements thanother measurement methods (e.g., measurement data from photographiccamera data alone). The dimensions measured may include dimensions ofany object and/or wall of a room.

At step 408, the servers 110 build a 3D model of the house or part ofthe house (e.g., one or more rooms, etc.) based upon the deriveddimensions. The 3D model may further be based upon camera data fromcamera 125; preexisting home data from database 130; and/or drone datafrom drone 140. For instance, the model may include dimension data of anobject from the LIDAR data, and color data of the object from the cameradata.

In another aspect, structural data from database 130 may show where abeam needed for support is, and this may be depicted in the 3D model.For instance, the 3D model may show the wall with a cut out indicatingwhere the necessary support beam is; or, the necessary support beam maybe indicated by a different color that contrasts it from the wallcovering the necessary support beam.

In another aspect, drone data (e.g., a LIDAR camera, a photographiccamera, or any other data coming from a drone inside or outside of thebuilding) may aide in building the 3D model. For instance, a droneflying exterior to the house may provide data verifying where a windowor skylight is.

At step 410, the 3D model is displayed by visually navigating throughthe 3D model based upon navigation input received from a human user suchas a selection of a directional arrow on the 3D model. In someembodiments, the human user input is received from 3D glasses of thehuman user.

At optional step 412, dimensional data, such as room dimensions ordimensions of an object in the room, is overlaid onto the displayed 3Dmodel.

At optional step 414, a 2D image (e.g., a snapshot from the 3D model) isdisplayed. The 2D image may be displayed based upon a command from ahuman user. The displayed 2D image may include dimensional data, such asroom dimensions or dimensions of an object in the room.

FIG. 4B shows a flowchart of an exemplary computer-implemented method400B for representation of a home, including navigating through a 3Dmodel according to a default navigation generated by a machine learningprogram. FIG. 4B is somewhat similar to FIG. 4A, except that step 410from FIG. 4A is replaced by step 409. At step 409, the 3D model isdisplayed by visually navigating through the 3D model based upon adefault navigation generated by a machine learning algorithm.

In some embodiments, the machine learning algorithm used to generate thedefault navigation may be a supervised learning algorithm, employdecision trees, make use of an artificial neural network, make use ofBayesian statistical analysis, or combinations thereof. In this regard,a processor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning algorithm mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, or a combined learning module or programthat learns in two or more fields or areas of interest. Machine learningmay involve identifying and recognizing patterns in existing data inorder to facilitate making predictions for subsequent data. Models maybe created based upon example inputs in order to make valid and reliablepredictions for novel inputs.

It should be noted that some known 3D models for representing a homecomprise images placed on top of each other. Using LIDAR data forbuilding a 3D model for representing a home advantageously improves onthese known systems. For instance, LIDAR data provides more accuratedimensional data than the image data alone does.

Exemplary System for a 3D Model for Viewing Potential Placement of anObject

FIG. 5 shows an exemplary computer system for viewing potentialplacement of an object in a room. With reference thereto, servers 110are shown sending and receiving information with LIDAR camera 120. Thegathered LIDAR data may be analyzed to determine the dimensions of anobject (e.g., the length, width, and/or height of the object, etc.). TheLIDAR data may be used to create a partial or complete 3D model of theobject, and may include 3D point cloud(s) created from the LIDAR data.The LIDAR camera 120 may be operated by any human or machine.

In some embodiments, the LIDAR camera 120 may be operated by the ownerof an object. For instance, an individual who is considering purchasinga house may wish to see how the furniture in her current house wouldlook in the house she is considering purchasing. In accordance with thetechniques described herein, the individual may operate the LIDAR camera120 to generate LIDAR data of a piece of furniture of her current house.This LIDAR data may then be used to display a 3D model including thepiece of furniture in a room of the house the individual is consideringpurchasing.

In some embodiments, the LIDAR camera 120 may be operated by a companyemployee. For instance, an employee of a furniture store (e.g.,furniture store 510) may use the LIDAR camera to scan the furniturestore's inventory. The LIDAR data may then be used to measuredimensional data of objects (e.g., pieces of furniture) of the store'sinventory, which are then inserted into a 3D model of a room therebyallowing a customer of the furniture store to view an object in a room.

Somewhat analogously to the furniture store example, the LIDAR cameramay be operated by an employee of appliance store 520 thus allowingcustomers of the appliance store 520 to view appliances in a 3D model ofa room.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on homes, rooms, or objects. For instance, data may begathered from public records, property deeds, government records,realtors (e.g., from websites and apps that realtors post informationto), websites (e.g., websites that display information of houses forsale), previous insurance claims, and so forth. In some embodiments,this data is gathered from database 530. The database 530 may include 3Dmodels of homes or rooms; or may include enough information about a homeor room that the server 110 is able to construct a 3D model of the homeor room from the information.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Each server 110 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system for viewing potential placement of an object 500, in additionto other software applications. The server 110 may further include adatabase 146, which may be adapted to store data related to the LIDARcamera 120, as well as any other data. The server 110 may access datastored in the database 146 when executing various functions and tasksassociated with LIDAR technology.

Although the exemplary system 500 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 5 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110 and/orany of the other elements shown in FIG. 5 may be utilized. For instance,the system 500 may include any number of servers 110 and hundreds ofmobile LIDAR cameras 120 or drones 140.

Furthermore, the database storage or processing performed by the one ormore servers 110 may be distributed among a plurality of servers 110 inan arrangement known as “cloud computing.” This configuration mayprovide various advantages, such as enabling near real-time uploads anddownloads of information as well as periodic uploads and downloads ofinformation.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a database application 540for data from a database; camera data receiving application 144 forreceiving camera data; a 3D model builder/modifier 145 for building andmodifying a 3D model (e.g., of a home, room, object etc.); a displayapplication 146 for displaying a representation of the 3D model; anavigation input receiver 147 for receiving navigation input; and/orobject inserting application 541 for building and inserting arepresentation of an object into a 3D model. The various softwareapplications may be executed on the same computer processor or ondifferent computer processors.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110. The machine learning algorithm may be asupervised learning algorithm, employ decision trees, make use of anartificial neural network, make use of Bayesian statistical analysis, orcombinations thereof. In this regard, a processor or a processingelement may be trained using supervised or unsupervised machinelearning, and the machine learning algorithm may employ a neuralnetwork, which may be a convolutional neural network, a deep learningneural network, or a combined learning module or program that learns intwo or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

It should be understood that, over time, the servers 110 may accumulatea large pool of data on objects (e.g., furniture, appliances, etc.).

The data described above may be used (e.g., with a machine learningalgorithm described above or by any other technique) to display arepresentation of the object on a 3D model. The machine learningalgorithm may be trained using any previously known data.

Exemplary Computer-Implemented Method for Viewing Potential Placement ofan Object

FIG. 6 shows a flowchart of an exemplary computer-implemented method 600for viewing potential placement of an object in a room. With referencethereto, at step 602, LIDAR data (e.g., from LIDAR camera 120) isreceived, such as via wireless communication or data transmission overone or more radio frequency links, by server 110. In some embodiments,the LIDAR data is transferred to the servers 110 from the LIDAR camera120 via a physical memory device.

At optional step 604, the server 110 receives any of: camera data fromcamera 125; preexisting home data from database 530; and/or drone datafrom drone 140. At step 606, the servers 110 measure a plurality ofdimensions of the object based upon processor analysis of the LIDARdata. In this respect, LIDAR data provides more accurate measurementsthan other measurement methods (e.g., measurement data from photographiccamera data alone). The dimensions measured may include any dimensionsof the object.

Exemplary objects include furniture such as chairs, couches, tables,desks and/or lamps. Further exemplary objects include appliances such asrefrigerators, stoves, microwaves, dishwashers, air fryers, laundrymachines, and/or dryers.

At step 608, the servers 110 receive or generate a 3D model of a room.The 3D model may include dimensional data of the room. The 3D model maybe received from a database such as database 530, or received from anyother source. Alternatively, the model may be generated by the servers110 from information received from database 530, or received from anyother source.

At step 610, a representation of the object is inserted into the 3Dmodel of the room based upon processor analysis of: (i) the plurality ofdimensions of the object measured from the LIDAR data; and (ii) thedimensional data of the room. The representation of the object mayfurther be based upon camera data from camera 125; data from database530; and/or drone data from drone 140. For instance, the model mayinclude dimension data of an object from the LIDAR data, and color dataand/or transparency data of the object from the camera data.

At step 612, a 3D model of the room is displayed with the insertedrepresentation of the object. In some embodiments, the 3D model isdisplayed on a pair of computerized glasses.

At optional step 614, dimensional data, such as room dimensions ordimensions of an object in the room, is overlaid onto the displayed 3Dmodel. For instance, the height and width of an object may be overlaidonto the representation of the object. Furthermore, price data of theobject may also be overlaid onto the object.

At optional step 616, a 2D image (e.g., a snapshot from the 3D model) isdisplayed. The 2D image may be displayed based upon a command from ahuman user. The displayed 2D image may include dimensional data, such asroom dimensions or dimensions of the object in the room.

At optional step 618, a warning is generated if the object will not fitinto the room. For instance, based upon dimensional data of the 3D modelof the room, the servers 110 may determine that a couch (e.g., theobject) will not fit through the only doorway to the room, and thuscannot be placed inside the room. In another example, the servers 110may determine that a refrigerator is taller than a ceiling of the room,and thus cannot be properly placed in the room.

In optional step 620, the representation of the object in the 3D modelis moved according to a command from a human user.

It should be understood that steps of the exemplary process 500 may berepeated such that many objects are scanned with the LIDAR camera 120.Thus, in some embodiments, a user may be presented with a list ofobjects, and the user may make a selection from the list so that thesystem displays the selected object. In this regard, is desired, theuser may go through the list of objects one by one. This list may alsoinclude prices of the objects.

Exemplary System for AI Based Recommendations for Object Placement in aHome

FIG. 7 shows an exemplary computer system for AI based recommendationsfor object placement in a home. With reference thereto, each server 110may include one or more computer processors adapted and configured toexecute various software applications and components of the system forAI based recommendations for object placement in a home. In particular,the server 110 may include virtual interior designer 741 (located, e.g.,on program memory 160).

In some embodiments, the virtual interior designer 741 comprises amachine learning algorithm for recommending object placement in a home.Exemplary types of objects that the machine learning algorithm mayprovide a recommended placement for are: chairs, tables, desks, couches,lamps, bookshelves, pictures, paintings, etc. However, the machinelearning algorithm may provide a recommendation for placement of anykind of object.

The machine learning algorithm may be trained using preexisting data(e.g., from database 730, or database 146, etc.). The preexisting datamay include data of object placement in rooms. In some embodiments, thispreexisting data is generated from analysis of home and/or realitywebsites (e.g., online websites used to advertise houses for sale). Insome embodiments, the preexisting data may be generated from smartdevices 710 of homeowners (e.g., smartphones, security cameras, etc.).In some embodiments, the preexisting data may be generated by LIDARcamera 120, photographic camera 125, and/or drone 140.

The machine learning algorithm may be used to analyze any or all of thedata held by servers 110 and/or database 730. The machine learningalgorithm may be a supervised learning algorithm, employ decision trees,make use of an artificial neural network, make use of Bayesianstatistical analysis, or combinations thereof. In this regard, aprocessor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning algorithm mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, or a combined learning module or programthat learns in two or more fields or areas of interest. Machine learningmay involve identifying and recognizing patterns in existing data inorder to facilitate making predictions for subsequent data. Models maybe created based upon example inputs in order to make valid and reliablepredictions for novel inputs. In some embodiments, the machine learningalgorithm is a convolutional neural network (CNN); in some embodiments,the machine learning algorithm is a deep neural network (DNN); in someembodiments, the machine learning algorithm is a recurrent neuralnetwork (RNN). It is advantageous to use machine learning algorithmsthat are particularly adept at finding patterns (e.g., CNNs).

Although the exemplary system 700 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 7 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110, and/orany of the other elements shown in FIG. 7 may be utilized. For instance,the system 700 may include any number of servers 110 and hundreds ofmobile LIDAR cameras 120 or drones 140.

Furthermore, the database storage or processing performed by the one ormore servers 110 may be distributed among a plurality of servers 110 inan arrangement known as “cloud computing.” This configuration mayprovide various advantages, such as enabling near real-time uploads anddownloads of information as well as periodic uploads and downloads ofinformation.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a receiving preexisting housedata application 143 for receiving preexisting data; camera datareceiving application 144 for receiving camera data; a 3D modelbuilder/modifier 145 for building and modifying a 3D model (e.g., of ahome, room, object etc.); a display application 146 for displaying arepresentation of the 3D model; a navigation input receiver 147 forreceiving navigation input; and/or the virtual interior designer 741 forgenerating a recommendation of an object placement. The various softwareapplications may be executed on the same computer processor or ondifferent computer processors.

Furthermore, in the exemplary system of FIG. 7 , the servers 110 areshown sending and receiving information with LIDAR camera 120. Thegathered LIDAR data may be analyzed to determine the dimensions of anobject (e.g., length, width, height, curvature, etc.).

The LIDAR data may also be used to determine the dimensions of a house,including dimensions of part or all of the interior and exterior of thehouse, as well as specific dimensions of any part of a room (e.g., thelength, width, and/or height of a wall, or of an object in the room,etc.). The LIDAR data may be used to create a partial or complete homemodel, and/or a partial or complete room model. Furthermore, 3D pointcloud(s) may be created from the LIDAR data. The LIDAR camera 120 may beoperated by any human or machine.

In some embodiments, the LIDAR camera 120 may be operated by the ownerof an object. For instance, the owner of a bookshelf may wish to beprovided with a recommendation for placement of the bookshelf in acertain room. In accordance with the techniques described herein, thebookshelf owner may use the LIDAR camera 120 to obtain information ofthe bookshelf and/or the room that it is desired to place the bookshelfin. A machine learning algorithm may then use this information toprovide a recommendation for placement of the bookshelf.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the home. For instance, data may be gathered frompublic records, property deeds, government records, realtors (e.g., fromwebsites and apps that realtors post information to), websites (e.g.,websites that display information of houses for sale), previousinsurance claims, and so forth. In some embodiments, this data isgathered from preexisting home database 730. In some embodiments, thedatabase 730 may include structural data of the home.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Exemplary Computer-Implemented Method for AI Based Recommendations forObject Placement in a Home

FIG. 8 shows a flowchart of an exemplary computer-implemented method 800for AI based recommendations for object placement in a home. Withreference thereto, at step 802, a machine learning algorithm, such as aCNN, DNN, or RNN, is trained. The machine learning algorithm may betrained based upon any data, such as data of object placement in a room.This data may be gathered from, for example, database 730 and/ordatabase 146.

At step 804, the system receives room data. The room data may comprisedimensional data of the room (e.g., length, width, and/or height of theroom), color data of the room, and other data of the room (e.g., window,door, and/or skylight placement in the room; etc.).

At step 806, object data is received. The object data may comprise:dimensional data of the object, type of the object, and/or color data ofthe object, etc. The object data may be received from any source(s)(e.g., databases, websites, LIDAR camera(s), photographic camera(s),drone(s), etc.). In some embodiments, dimensional data of an object maybe received from a LIDAR camera(s), and color data of the object may bereceived from photographic camera(s). In this regard, in someembodiments, the object may be a painting or photograph, and the colordata of the painting or photograph may be from a photographic camera,and the dimensional data of the photograph may be from a LIDAR camera.

At optional step 808, a user sends, to the system, a placement of afirst item in the room. The system may optionally (e.g., at step 812)use this placement of the first item in the room as part of determiningthe recommendation for the object placement. For instance, a user maylook at an empty room, which she intends to set up a desk (e.g., the“first item”) and a bookshelf (e.g., the “object”) in; the user may knowthat she intends to place the desk in a particular corner of the room,but would like to consult the program on where to place the bookshelf;thus, the user may enter, into the system, the intended placement of thedesk, and the system may then consider the desk placement when providinga recommendation for placement of the bookshelf.

At optional step 810, the system generates a user profile. The profilemay be generated based upon any information of the user. For instance,the user profile may be based upon a placement of objects in the user'scurrent house; in some embodiments, the user may send photographs (e.g.,through an app or website) of the users current house to the system sothat the system may determine the object placement in the user's currenthouse. Additionally or alternatively, in some embodiments, the userprofile may be based upon a survey given by the system to the user; thesurvey may be given in the form of displaying object placements in aroom, and receiving selections of preferred object placements from theuser.

At step 812, using the trained machine learning algorithm, the systemgenerates a recommendation for placement of the object in the room basedupon: (i) the received room data, (ii) the received object data, (iii)the placement of the first item received from the user, and/or (iv) thegenerated user profile. At step 814, the recommendation for placement ofthe object is displayed on a display device.

At optional step 816, the system generates additional recommendations(e.g., based upon: (i) the received room data, (ii) the received objectdata, (iii) the placement of the first item received from the user,and/or (iv) the generated user profile), and displays therecommendations as options for the user to select from.

Exemplary System for Visualization of Landscape Design

FIG. 9 shows an exemplary computer system for visualization of landscapedesign. With reference thereto, servers 110 are shown sending andreceiving information with LIDAR camera 120. The gathered LIDAR data maybe analyzed to determine the dimensions of a landscape, such as thedimensions of a landscape surrounding a house, as well as any objects inthe landscape. The LIDAR data may be used to create a partial orcomplete landscape model. Furthermore, 3D point cloud(s) may be createdfrom the LIDAR data.

The LIDAR camera 120 may be operated by any human or machine. In someembodiments, the LIDAR camera 120 may be operated by a homeowner. Forinstance, a home owner may wish to sell his home. In accordance with thetechniques described herein, the homeowner may operate LIDAR camera 120to generate data of the landscape, which may be used to create a 3Dmodel of the landscape and further used to visualize the landscape(e.g., for advertisement on a real estate website). Additionally oralternatively, the system may provide the homeowner with arecommendation for placement of an object in the landscape.

In some embodiments, the LIDAR camera 120 may be operated by a realestate agent or a company employee (e.g., an employee of an onlinewebsite for marketing houses). For instance, a real estate agent or acompany employee may bring a LIDAR camera to an individual's property,and use the LIDAR camera 120 to gather LIDAR data from the landscapeand/or home on the landscape. The LIDAR data may then be used to createa 3D model of the landscape, and display the 3D model on an onlinewebsite.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the landscape and/or home. For instance, data may begathered from public records, property deeds, government records,realtors (e.g., from websites and apps that realtors post informationto), websites (e.g., websites that display information of houses forsale), previous insurance claims, and so forth. In some embodiments,this data is gathered from database 930. In some embodiments, thedatabase 930 may also include structural data of the home.

Although the exemplary system 900 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 9 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110, and/orany of the other elements shown in FIG. 9 may be utilized. For instance,the system 900 may include any number of servers 110 and hundreds ofmobile LIDAR cameras 120 or drones 140.

Furthermore, the database storage or processing performed by the one ormore servers 110 may be distributed among a plurality of servers 110 inan arrangement known as “cloud computing.” This configuration mayprovide various advantages, such as enabling near real-time uploads anddownloads of information as well as periodic uploads and downloads ofinformation.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a receiving preexisting housedata application 143 for receiving preexisting data; camera datareceiving application 144 for receiving camera data; a 3D modelbuilder/modifier 945 for building and modifying a 3D model (e.g., of alandscape); a display application 146 for displaying a representation ofthe 3D model; a navigation input receiver 147 for receiving navigationinput; and a landscape designer 941 for generating a recommendation ofan object placement. The various software applications may be executedon the same computer processor or on different computer processors.

In some embodiments, the landscape designer 941 comprises a machinelearning algorithm for recommending object placement on a landscape.Exemplary types of objects that the machine learning algorithm mayprovide a recommended placement for are: patios, sheds, garages, fences,trees, plants, flowers, and/or pathways, etc. However, the machinelearning algorithm may provide a recommendation for placement of anykind of object.

The machine learning algorithm may be trained using preexisting data(e.g., from database 930, or database 146, etc.). The preexisting datamay include data of object placement in landscapes (e.g., from images).In some embodiments, the preexisting data is generated from images oflandscapes found on landscaping websites. Additionally or alternatively,this preexisting data is generated from analysis of home and/or realitywebsites (e.g., online websites used to advertise houses for sale). Insome embodiments, the preexisting data may be generated from smartdevices 710 of homeowners (e.g., smartphones, security cameras, etc.).In some embodiments, the preexisting data may be generated by LIDARcamera 120, photographic camera 125, and/or drone 140.

The machine learning algorithm may be used to analyze any or all of thedata held by servers 110 and/or database 930. The machine learningalgorithm may be a supervised learning algorithm, employ decision trees,make use of an artificial neural network, make use of Bayesianstatistical analysis, or combinations thereof. In this regard, aprocessor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning algorithm mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, or a combined learning module or programthat learns in two or more fields or areas of interest. Machine learningmay involve identifying and recognizing patterns in existing data inorder to facilitate making predictions for subsequent data. Models maybe created based upon example inputs in order to make valid and reliablepredictions for novel inputs. In some embodiments, the machine learningalgorithm is a convolutional neural network (CNN); in some embodiments,the machine learning algorithm is a deep neural network (DNN); in someembodiments, the machine learning algorithm is a recurrent neuralnetwork (RNN). It is advantageous to use machine learning algorithmsthat are particularly adept at finding patterns (e.g., CNNs).

Exemplary Computer-Implemented Method for Visualization of LandscapeDesign

FIG. 10 shows a flowchart of an exemplary computer-implemented method1000 for viewing potential placement of an object in a room. Withreference thereto, at step 1002, LIDAR data (e.g., from LIDAR camera120) is received, such as via wireless communication or datatransmission over one or more radio frequency links, by server 110. Insome embodiments, the LIDAR data is transferred to the servers 110 fromthe LIDAR camera 120 via a physical memory device.

At optional step 1004, the server 110 receives any of: camera data fromcamera 125; smart device data from smart devices 710; data from database930; and/or drone data from drone 140. At step 1006, the servers 110measure a plurality of dimensions of the landscape and/or object(s) onthe landscape based upon processor analysis of the LIDAR data. In thisrespect, LIDAR data provides more accurate measurements than othermeasurement methods (e.g., measurement data from photographic cameradata alone). The dimensions measured may include any dimensions of thelandscape and/or object on the landscape.

At step 1008, a structure and/or vegetation is determined in thelandscape. In some embodiments, the structure is a home on thelandscape. The structure and/or vegetation may be determined from any ofthe LIDAR data, the camera data, the smart device data, the databasedata (e.g., from database 930) and/or the drone data. In someembodiments, the structure and/or vegetation is determined by using amachine learning algorithm in conjunction with the above-mentioned data.

At step 1010, the system builds a 3D model of the landscape based uponthe measured plurality of dimensions. The 3D model may further be basedupon the camera data, the smart device data, the database data (e.g.,from database 930) and/or the drone data. The 3D model may include thestructure and/or the vegetation.

At step 1012, a representation of the 3D model is displayed.

At optional step 1014, dimensional data may be displayed. For instance,dimensional data of the property boundaries may be overlaid onto thedisplayed 3D model. Additionally or alternatively, dimensional data ofan object (e.g., a shed, garage, house, etc.) may be displayed. In someembodiments, the dimensional data may be determined from any of theLIDAR data, the camera data, the smart device data, the database data(e.g., from database 930) and/or the drone data.

At optional step 1016, a 2D image (e.g., a snapshot from the 3D model)is displayed. The 2D image may be displayed based upon a command from ahuman user. The displayed 2D image may include dimensional data, such asthe dimensions mentioned above with respect to step 1014.

At optional step 1018, the system generates a recommendation forplacement of an object in the landscape. In some embodiments, asmentioned above, the recommendation may be provided by a machinelearning algorithm of the landscape designer 941.

At optional step 1020, the representation of an object in the 3D modelis moved according to a command from a human user. For instance, a usermay be considering where to build a shed on the landscape, and use thisoptional step to view different potential placements of the shed.

Exemplary System for Visualization of a Utility Line

FIG. 11 shows an exemplary computer system for visualization of autility line. With reference thereto, servers 110 are shown sending andreceiving information with LIDAR camera 120. The gathered LIDAR data maybe analyzed in conjunction with other data (e.g., from public recordsdatabase 1130) to determine a location of a utility line (e.g., a powerline, a water line, a gas line, a cable line, a fiber optic line, etc.).

For instance, the LIDAR data may be used to determine the location anddimensions of a house. Then, the determined location and dimensions ofthe house may be matched with data from a public records databaseincluding data showing a location of a utility line. The matching ofthis data advantageously allows for simple and precise marking ordetermination of a location of a utility line. For instance, previousmethods required a utility company employee to come to the property tomark the location of the utility line either with paint or small flags.However, with the systems and methods described herein, a homeownercould—simply with a LIDAR camera and smartphone—determine the locationof a utility line, and there would be no need for the utility companyemployee to come to the property. This is quite advantageous when thehome owner wishes to put a sign (e.g., a “for sale” sign) on theproperty, or when the homeowner wishes to dorepairs/renovations/additions to the home that require knowledge of thelocations of the utility lines.

The LIDAR camera 120 may be operated by any human or machine. In someembodiments, the LIDAR camera 120 may be operated by the homeowner. Insome embodiments, the LIDAR camera 120 may be operated by a companyemployee, such as an employee of a construction company building anaddition onto a house. In some embodiments, the LIDAR camera 120 isoperated by an employee of a company placing a for sale sign on a yard.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the landscape and/or home. For instance, data may begathered from public records, property deeds, government records,realtors (e.g., from websites and apps that realtors post informationto), websites (e.g., websites that display information of houses forsale), previous insurance claims, and so forth. In some embodiments,this data is gathered from public records database 1130. The publicrecords database 1130 may include structural data of the home.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Each server 110 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system for a 3D home model for visualization of utility lines 1100,in addition to other software applications. The server 110 may furtherinclude a database 146, which may be adapted to store data related tothe LIDAR camera 120, as well as any other data. The server 110 mayaccess data stored in the database 146 when executing various functionsand tasks associated with LIDAR technology.

Although the exemplary system 1100 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 11 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110, and/orany of the other elements shown in FIG. 11 may be utilized. Forinstance, the system 1100 may include any number of servers 110 andhundreds of mobile LIDAR cameras 120 or drones 140. Furthermore, thedatabase storage or processing performed by the one or more servers 110may be distributed among a plurality of servers 110 in an arrangementknown as “cloud computing.” This configuration may provide variousadvantages, such as enabling near real-time uploads and downloads ofinformation as well as periodic uploads and downloads of information.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162. Similarly, thememory of the controller 155 may include multiple RAMs 164 and multipleprogram memories 160. Although the I/O circuit 166 is shown as a singleblock, it should be appreciated that the I/O circuit 166 may include anumber of different types of I/O circuits. The RAM 164 and programmemories 160 may be implemented as semiconductor memories, magneticallyreadable memories, or optically readable memories, for instance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a data receiving application1145 for receiving data such as preexisting utility line data; cameradata receiving application 144 for receiving camera data; a 3D modelbuilder/modifier 145 for building and modifying a 3D model (e.g., of alandscape surrounding a utility line etc.); a display application 146for displaying a representation of the 3D model; a navigation inputreceiver 147 for receiving navigation input; and/or a utility linedetermination application 1141 for determining the location of a utilityline. The various software applications may be executed on the samecomputer processor or on different computer processors.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110. The machine learning algorithm may be asupervised learning algorithm, employ decision trees, make use of anartificial neural network, make use of Bayesian statistical analysis, orcombinations thereof. In this regard, a processor or a processingelement may be trained using supervised or unsupervised machinelearning, and the machine learning algorithm may employ a neuralnetwork, which may be a convolutional neural network, a deep learningneural network, or a combined learning module or program that learns intwo or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

It should be understood that, over time, the servers 110 may accumulatea large pool of data on an individual home or a group of homes.

The data described above may be used (e.g., with a machine learningalgorithm described above or by any other technique) to determine and/orvisualize the location of a utility line.

Exemplary Computer-Implemented Method for Visualization of a UtilityLine

FIG. 12 shows a flowchart of an exemplary computer-implemented method1200 for viewing potential placement of an object in a room. Withreference thereto, at step 1202, LIDAR data (e.g., from LIDAR camera120) is received, such as via wireless communication or datatransmission over one or more radio frequency links, by server 110. Insome embodiments, the LIDAR data is transferred the servers 110 from theLIDAR camera 120 via a physical memory device.

At step 1204, the system receives preexisting utility line data (e.g.,from a public records database or from any other source). In someembodiments, the preexisting utility line data is part of geographicdata of an area proximate to the utility line, and the geographic dataof the area proximate to the utility line also includes data of astructure in the area proximate to the utility line.

At optional step 1206, the system receives any of: camera data fromcamera 125; smart device data from smart devices 710; data from database1130; and/or drone data from drone 140.

At step 1208, the system determines a location of the utility line basedupon: the LIDAR data, the preexisting utility line, the data cameradata, the preexisting home data, the smart device data and/or the dronedata. In some embodiments, this is done by matching the LIDAR data withthe preexisting utility line data. For instance, the LIDAR data mayindicate that there is an object, such as the home, a tree, etc., at aparticular location.

The location of the object determined from the LIDAR data may then bematched with a location of the object from the preexisting utility linedata (e.g., a map of the property including locations of the utilitylines and locations of objects). Based upon this matching, the systemthen may know that the utility line is a certain distance and directionfrom the object, and thereby is able to provide the user with anindication of the location of the utility line.

At step 1210, the system displays a location of the utility line (e.g.,on a user's smartphone or on any other device). In this regard, a usermay hold a display device, such as the display on a smartphone, towardsthe ground, and receive an indication of the location of the utilityline. For instance, in real time, while the smartphone is pointedtowards the ground, the smartphone may show images or video of theground (e.g., from a camera of the smartphone), and overly the locationof the utility line onto the images or video of the ground.

At optional step 1212, the system may determine that there is a mistakein a public record. In one aspect, this is made possible because of thehigh accuracy of LIDAR data. For instance, the system may use the LIDARdata to determine that the location of a home, garage, and/or shed isslightly incorrect in a property record. The system may determine that ahouse is one foot larger or smaller than indicated in a property record.In another example, the system may determine that a house is closer toor farther away from a property boundary as indicated by a propertyrecord. This is made possible by the high accuracy of the LIDAR data.

Exemplary System for Commercial Inventory Mapping

FIG. 13 shows an exemplary computer system for commercial inventorymapping. With reference thereto, servers 110 are shown sending andreceiving information with LIDAR camera 120. The gathered LIDAR data maybe analyzed in conjunction with other data (e.g., from camera 125, drone14, database 1330, etc.) to create or add to a commercial inventory.

For instance, the LIDAR data may be used to determine dimensional dataof an object and/or a type of the object. The object may be any kind ofobject. For instance, the object may be any type including an objectsold online, an object sold in a “brick and mortar” store, an objecthoused in a warehouse, etc. In some embodiments, the object may be anobject commercially for sale. For instance, the object may be anelectronics item, a grocery item, a furniture item, a vehicle, exerciseequipment, etc.

In some embodiments, the LIDAR data is analyzed to determine thedimensional data of the object, and the type of object is thendetermined wholly or partially from the dimensional data. For instance,the object type may further be determined based upon photographic cameradata, data from database 1330, drone data, and/or data from smartdevices. In some embodiments, the type of the object is furtherdetermined based upon barcode data or quick response (QR) code datafound in the photographic camera data.

The LIDAR camera 120 may be operated by any human or machine. In someembodiments, the LIDAR camera 120 may be operated by an employee of acompany, such as a company that seeks to create an inventory ofcommercial items.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet.

In another example, the LIDAR data may be transferred from the LIDARcamera 120 to a computer (via, e.g., a cable, a USB device, or any othermeans), and then sent from the computer to the servers 110 by anymethods (e.g., sent by the internet, by Ethernet connection, or soforth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from sources that havedata on a commercial inventory, such as database 1330. For instance,database 1330 may store commercial inventory lists that the gatheredLIDAR data is used to add to or correct.

The servers 110 may also gather data from a drone 140 or a number ofdrones. Such data may include data from a camera on the drone, a LIDARcamera on the drone, radio detection and ranging (RADAR) data gatheredby the drone, global positioning system (GPS) data gathered by thedrone, information from an infrared camera of the drone, and so forth.

Each server 110 may include one or more computer processors adapted andconfigured to execute various software applications and components ofthe system for commercial inventory mapping 1300, in addition to othersoftware applications. The server 110 may further include a database146, which may be adapted to store data related to the LIDAR camera 120,as well as any other data. The server 110 may access data stored in thedatabase 146 when executing various functions and tasks associated withLIDAR technology.

Although the exemplary system 1300 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 13 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110, and/orany of the other elements shown in FIG. 13 may be utilized. Forinstance, the system 1300 may include any number of servers 110 andhundreds of mobile LIDAR cameras 120 or drones 140.

Furthermore, the database storage or processing performed by the one ormore servers 110 may be distributed among a plurality of servers 110 inan arrangement known as “cloud computing.” This configuration mayprovide various advantages, such as enabling near real-time uploads anddownloads of information as well as periodic uploads and downloads ofinformation.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a data receiving application1345 for receiving data such as inventory data; camera data receivingapplication 144 for receiving camera data; a 3D model builder/modifier145 for building and modifying a 3D model (e.g., of an object to beincluded in a commercial inventory etc.); a display application 146 fordisplaying a representation of the 3D model; a navigation input receiver147 for receiving navigation input; and/or a commercial inventorydetermination application 1341 for determining an inventory. The varioussoftware applications may be executed on the same computer processor oron different computer processors.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110. The machine learning algorithm may be asupervised learning algorithm, employ decision trees, make use of anartificial neural network, make use of Bayesian statistical analysis, orcombinations thereof. In this regard, a processor or a processingelement may be trained using supervised or unsupervised machinelearning, and the machine learning algorithm may employ a neuralnetwork, which may be a convolutional neural network, a deep learningneural network, or a combined learning module or program that learns intwo or more fields or areas of interest. Machine learning may involveidentifying and recognizing patterns in existing data in order tofacilitate making predictions for subsequent data. Models may be createdbased upon example inputs in order to make valid and reliablepredictions for novel inputs.

The data described above may be used (e.g., with a machine learningalgorithm described above or by any other technique) to create and/oradd to a commercial inventory list.

Exemplary Computer-Implemented Method for Commercial Inventory Mapping

FIG. 14A shows a flowchart of an exemplary computer-implemented method1400 for viewing potential placement of an object in a room. Withreference thereto, at step 1402, LIDAR data (e.g., from LIDAR camera120) is received, such as via wireless communication or datatransmission over one or more radio frequency links, by server 110. Insome embodiments, the LIDAR data is transferred to the servers 110 fromthe LIDAR camera 120 via a physical memory device.

At optional step 1404, the system receives additional data that may beused to augment the LIDAR data. The additional data may include, forinstance, camera data from camera 125, inventory data or other data fromdatabase 1330, drone data from drone 140, smart device data from smartdevice 710, and so forth.

At step 1406, data of an object is determined based upon processoranalysis of the LIDAR data. The data of the object may include: (i)dimensional data of the object, and (ii) a type of the object. Thedetermination of data of the object (especially the type of object) mayfurther be made based upon any or all of the data gathered at step 1404.It should be understood that object(s) may be found in LIDAR data usingmachine learning algorithms, such as a CNN, DNN, etc., or by any othertechnique.

The determined data of the object may also include a location of theobject, which may be determined by any suitable technique. For instance,the location of an object may be determined from the LIDAR data alone(e.g., processor analysis determines that an object is five feet from awall). In some embodiments, the LIDAR data is married with other data todetermine the object's location (e.g., the LIDAR data is married withcamera data, GPS data, RADAR data, drone data, bar code data, quickresponse (QR) code data, radio frequency ID (RFID) tag data, data of thelayout of a building, preexisting data of the object's location, etc.).

At step 1408, based upon the determined object data, the object is addedto an existing inventory list or used to create a new inventory list.

At step 1410, the system searches for an additional object (e.g., asecond object) in the LIDAR data. If there is a second object in theLIDAR data, the system returns to step 1406, and the system determinesobject data of the additional object.

If no additional object is found, the system proceeds to step 1412,where the system determines if it is finished creating the inventory. Ifthe system is not finished creating the inventory, additional LIDAR datais gathered at step 1414 (e.g., a company employee scans the next objectwith LIDAR camera 120). And, optionally at step 1416, somewhat similarlyto step 1404, additional data used to augment the additional LIDAR datais received. The system then proceeds/returns to step 1406 where data ofan object from the additional LIDAR data is determined.

If the system is finished creating the inventory at step 1412, thesystem then optionally creates a map detailing locations of objects inthe inventory. It should be understood that the method of FIG. 14A isonly an example, and, in some embodiments, the system may create the mapeven if the system is not finished creating the inventory. At optionalstep 1420, the map may be used to provide navigation instructions to,for example, a customer or employee of a store, or a customer oremployee of a warehouse. For instance, the system may display the mapwith the route to a particular item indicated on the map. In someembodiments, the system provides an aisle number or other indicator ofthe item's location to the user.

FIG. 14B shows a flowchart of another exemplary computer-implementedmethod for commercial inventory mapping 1425. The computer-implementedmethod 1425 may be implemented via one or more local or remoteprocessors, servers, transceivers, sensors, and/or LIDAR devices. Themethod 1425 may include generating a LIDAR-based or other virtual map ofa store 1426. For instance, a virtual map of a store may be completelybased upon LIDAR data. Additionally or alternatively, a high levelvirtual map of a store may be generated, and more detailed LIDAR datamay be used to refine the accuracy of the high level virtual map of thestore.

The computer-implemented method 1425 may include overlaying, via one ormore local or remote processors, customer aisle flow on top of thevirtual LIDAR map of the store 1428. For instance, grocery stores,department stores, hardware stores, restaurants, and other places ofbusiness may prefer that customers enter and exit certain doors into andout of the store, and request that customers and employees followcertain flows when moving about the store, such as to allow for socialdistancing and/or to allow customers minimize their time within thestore.

The computer-implemented method 1425 may include determining, via one ormore local or remote processors, a location of each good or grocery itemwithin a store 1430. For instance, LIDAR or other imaging techniques maybe used to gather data on each item within a store, identify the type ofitem, and identify the location of each item. The location of each itemmay include an aisle of the item, a shelf or row of the item, a bin ofthe item, an area of the store in which the item is located, or otherlocation of the item within the store. Additionally or alternatively,sensors and/or cameras may be used to determine the location of eachgood or grocery item within a store. For instance, each good may haveits own chip/processor and/or sensor, such as a GPS (Global PositioningSystem) unit, RFID tag, or other type of sensor or transceiver tocommunicate with other processors or nodes about a store. Additionallyor alternatively, each shelf or bay within a store may have its ownsensor(s) and/or camera(s) to facilitate good identification and/orlocationing.

The computer-implemented method 1425 may include adding, via one or morelocal or remote processors, the location of each item within the storeto the LIDAR-based or other virtual map of the store 1432. For instance,the method 1425 may include populating or overlaying the location ofeach item on the LIDAR-based or other virtual map of the store.

The computer-implemented method 1425 may include generating, via one ormore local or remote processors, an App for download that includes theLIDAR-based or other virtual map of the store 1434, along with locationsof items throughout the store, including icons designating each type ofitem (such as icons representing grocery/food items—chicken, bacon,eggs, milk, cat food, etc., or icons representing other goods—such astools, paint, plants, gloves, saws, clothing, socks, laundry detergent,paper towels, etc.).

The computer-implemented method 1425 may include receiving, such as at aremote or local server associated with a store, a virtual order from amobile device of a customer on which the App is installed 1436. Thevirtual order may be a virtual order for a list of groceries or othergoods (such as clothing, tools, etc.), for example.

The computer-implemented method 1425 may include verifying, via one ormore local or remote processors, that one or more of items on thecustomer's virtual order or list of items are currently available in thestore 1438. For instance, a processor may update the list of itemscurrently in the store every morning, and then compare the customer'slist of items with the updated list of items, and also with othercustomer orders and purchases throughout the day to provide real timeavailability. Once an item is purchased, the processor may blockadditional orders of the item by other customers, or otherwise reservethe item for the customer that purchased the item first.

The computer-implemented method 1425 may include generating, via one ormore local or remote processors, a route through the store for thecustomer, or an employee to follow (for instance, to facilitatecurb-side pick-up or delivery to the customer's location), to pick upall of the items in the store that are currently available and tosatisfy pre-determined flow through the store 1440. For instance, aislesmay be intended for one-way traffic or people movement to facilitatesocial distancing and/or minimizing time within the store.

The computer-implemented method 1425 may include generating, via one ormore local or remote processors, icons representing each item on thecustomer's list of groceries or other goods, and overlaying those iconson the LIDAR-based or other virtual map of the store 1442. Additionallyor alternatively, the computer-implemented method 1425 may includehighlighting, via one or more processors, the person flow through thestore, and/or highlighting the location and/or location of where eachitem on the customer's list is located within the store 1444.

The computer-implemented method 1425 may include charging, via one ormore local or remote processors, a virtual pay account of the customerfor the goods/groceries, and/or confirm pick up or delivery of the goodsand/or groceries 1446. The method may include additional, less, oralternate actions, including those discussed elsewhere herein, includingthose actions discussed with respect to FIG. 14A above, or with respectto FIG. 14C below. Although the focus of the foregoing discussion hasbeen on groceries, the present embodiments also apply to other goods andstores, such as clothing and department or retail stores.

FIG. 14C shows a flowchart of another exemplary computer-implementedmethod for commercial inventory mapping 1450. The computer-implementedmethod 1450 may be implemented via one or more local or remoteprocessors, servers, transceivers, sensors, and/or LIDAR devices. Themethod 1450 may include generating a LIDAR-based or other virtual map ofa store 1452. For instance, a virtual map of a store may be completelybased upon LIDAR data. Additionally or alternatively, a high levelvirtual map of a store may be generated, and more detailed LIDAR datamay be used to refine the accuracy of the high level virtual map of thestore.

The computer-implemented method 1450 may include determining, via one ormore local or remote processors, a location of each good or grocery itemwithin a store 1454. For instance, LIDAR or other imaging techniques maybe used to gather data on each item within a store, identify the type ofitem, and identify the location of each item. The location of each itemmay include an aisle of the item, a shelf or row of the item, a bin ofthe item, an area of the store in which the item is located, or otherlocation of the item within the store. Additionally or alternatively,sensors and/or cameras may be used to determine the location of eachgood or grocery item within a store. For instance, each good may haveits own chip/processor and/or sensor, such as a GPS (Global PositioningSystem) unit, RFID tag, or other type of sensor or transceiver tocommunicate with other processors or nodes about a store. Additionallyor alternatively, each shelf or bay within a store may have its ownsensor(s) and/or camera(s) to facilitate good identification and/orlocation identification. The location of each item may also be performedby manually scanning the RFID tag on each item, and sending the data toan inventory database.

The computer-implemented method 1450 may include updating ormaintaining, via one or more local or remote processors, inventoryinformation and/or good location and availability information 1456. Forinstance, an inventory database may be maintained, and as ordersreceived and/or confirmed, the inventory of goods may be updated.

The computer-implemented method 1450 may include receiving, via one ormore local or remote processors, a virtual or electronic customer order1458 via wireless communication or data transmission over one or moreradio frequency links. For instance, a customer may enter an order forgroceries via an App installed on their mobile device, and transmit theorder via their mobile device to a server associated with the respectivestore for processing and virtual map generation.

The computer-implemented method 1450 may include determining, via one ormore local or remote processors, whether the desired goods on thecustomer's virtual list of goods are still available within the store'sinventory 1460, and if so, then reserving the goods that are availablefor the customer and/or otherwise blocking orders from other customersfor the desired goods.

The computer-implemented method 1450 may include generating, via one ormore local or remote processors, icons representing each item on thecustomer's list of groceries or other goods that are currentlyavailable, and overlaying those icons on the LIDAR-based or othervirtual map of the store 1462. Additionally or alternatively, thecomputer-implemented method 1450 may include highlighting, via one ormore local or remote processors, the person flow through the store,and/or highlighting the location and/or location of where each availableitem on the customer's list is located within the store.

The computer-implemented method 1450 may include generating, via one ormore local or remote processors, a route through the store for thecustomer, or an employee to follow (for instance, to facilitatecurb-side pick-up or delivery to the customer's location), to pick upall of the items in the store that are currently available and tosatisfy pre-determined flow through the store 1464. For instance, aislesmay be intended for one-way traffic or people movement to facilitatesocial distancing and/or minimizing time within the store.

The computer-implemented method 1450 may include determining, via one ormore local or remote processors, transceivers, and/or sensors, that oneor more of the available items have been picked up by the customer 1466.For instance, good-mounted or shelf-mounted sensors may detect that anitem has moved, and/or cameras may detect that a customer has placed anitem in their cart. Additionally or alternatively, sensors at an exit ofthe store may automatically log the items in the customer's bag or cart,and automatically charge a financial account associated with thecustomer.

The computer-implemented method 1450 may include charging, via one ormore local or remote processors, a virtual pay account of the customerfor the goods/groceries, and/or confirm pick up or delivery of the goodsand/or groceries 1468. The method 1450 may include updating theelectronic inventory of the store and/or updating the inventory iconswithin the LIDAR-based virtual map of the store. The method may includeadditional, less, or alternate actions, including those discussedelsewhere herein, including those actions discussed with respect toFIGS. 14A & 14B above.

In one aspect, a computer-implemented method for commercial inventorymapping may be provided. The method may include, via one or more localor remote processors, sensors, servers, LIDAR devices, and/ortransceivers: (1) receiving light detection and ranging (LIDAR) datagenerated from a LIDAR camera, the LIDAR data associated with a store ora store layout; (2) generating a LIDAR-based virtual map of the storefrom processor analysis of the LIDAR data; (3) determining locations ofindividual goods in the store; (4) the locations of individual goodsonto the LIDAR-based virtual map; and/or (5) generating an updatedLIDAR-based virtual map of the store displaying aisles of the store andthe overlaid locations of the individual goods within the store. Themethod may include additional, less, or alternate actions, includingthose discussed elsewhere herein.

For instance, the method may include, via one or more local or remoteprocessors, sensors, servers, LIDAR devices, and/or transceivers: (a)overlaying customer flow through the store on top of the updatedLIDAR-based virtual map; and/or (b) displaying the updated LIDAR-basedvirtual map including the overlaid customer flow though the store on amobile device of the customer.

The method may include, via the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers, receiving avirtual or electronic customer list for items to be purchased from anApp installed on a customer mobile device via one or more radiofrequency links, and/or via wireless communication or data transmission.The method may also include, via the one or more local or remoteprocessors, sensors, servers, LIDAR devices, and/or transceivers,verifying that one or more of the items on the virtual or electroniccustomer list of items are currently available for purchase, such as byaccessing a list of current inventory within the store.

The method may include, via the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers, verifying that oneor more of the items on the virtual or electronic list of items will beavailable for purchase when the customer arrives at the store at asubsequent hour, such as by comparing the virtual or electronic list ofitems with current electronic orders received from other customers. Themethod may also include, via one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers, generating avirtual route through the store on top of the updated LIDAR-basedvirtual map that displays locations of items on the virtual orelectronic customer list of items received from the customer's mobiledevice; and/or displaying the virtual route on top of the updatedLIDAR-based virtual map of store so as to depict the route for acustomer or employee to travel though the store, such as on a mobiledevice of the customer, and pick up the desired items.

The method may include, via one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers, further updatingthe updated LIDAR-based virtual map to depict a route through the storefor the customer or an employee to pick up one or more items on thecustomer's electronic list. The method may also include, via the one ormore local or remote processors, sensors, servers, LIDAR devices, and/ortransceivers, further updating the updated LIDAR-based virtual map todepict a route through the store for the customer or an employee to pickup one or more items on the customer's electronic list, and displayingthe route on top of the updated LIDAR-based virtual map of the store ona mobile device, wherein the route is highlighted and reflectspre-determined flow through the store and icons representing aisle orother locations of items on the customer's electronic list are alsovirtually depicted.

In another aspect, a computer system configured for commercial inventorymapping may be provided. The computer system may include one or morelocal or remote processors, sensors, servers, LIDAR devices, and/ortransceivers configured to: (1) receive light detection and ranging(LIDAR) data generated from a LIDAR camera, such as via wirelesscommunication or data transmission over one or more radio frequencylinks, the LIDAR data associated with a store or a store layout; (2)generate a LIDAR-based virtual map of the store from processor analysisof the LIDAR data; (3) determine locations of individual goods in thestore; (4) overlay the locations of the individual goods onto theLIDAR-based virtual map; and/or (5) generate an updated LIDAR-basedvirtual map of the store displaying aisles of the store and the overlaidlocations of individual goods within the store. The computer system mayinclude additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the computer system and/or the one or more local or remoteprocessors, sensors, servers, LIDAR devices, and/or transceivers may beconfigured to: overlay customer flow through the store on top of theupdated LIDAR-based virtual map; and/or display the updated LIDAR-basedvirtual map including the overlaid customer flow though the store on amobile device of the customer or an employee.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto receive a virtual or electronic customer list for items to bepurchased from an App installed on a customer mobile device via one ormore radio frequency links, and/or via wireless communication or datatransmission.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto verify that one or more of the items on the virtual or electroniccustomer list of items are currently available for purchase, such as byaccessing a list of current inventory within the store.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto verify that one or more of the items on the virtual or electroniccustomer list of items will be available for purchase when the customerarrives at the store at a subsequent hour, such as by comparing thevirtual or electronic list of items with current electronic ordersreceived from other customers.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto generate a virtual route through the store on top of the updatedLIDAR-based virtual map that displays a location of each of the items onthe virtual or electronic customer list of items received from thecustomer's mobile device; and/or display the virtual route on top of theupdated LIDAR-based virtual map of store so as to depict the route for acustomer or employee to travel though the store, such as on a mobiledevice of the customer, and pick up the desired items.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto update the updated LIDAR-based virtual map to depict a route throughthe store for the customer or an employee to pick up one or more itemson the virtual or electronic customer list.

The computer system and/or the one or more local or remote processors,sensors, servers, LIDAR devices, and/or transceivers may be configuredto update the updated LIDAR-based virtual map to depict a route throughthe store for the customer or an employee to pick up one or more itemson the customer's electronic list, and display the route on top of theupdated LIDAR-based virtual map of the store on a mobile device, whereinthe route is highlighted and reflects pre-determined flow through thestore and icons representing aisle or other locations of items on thecustomer's electronic list are also virtually depicted.

In yet another aspect, there is a computer system configured forcommercial inventory mapping. The computer system may include: one ormore processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: (1) receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera viawireless communication or data transmission over one or more radiofrequency links, the LIDAR data associated with a store or a storelayout; (2) generate a LIDAR-based virtual map of the store fromprocessor analysis of the LIDAR data; (3) determine locations ofindividual goods in the store; (4) overlay the locations of theindividual goods onto the LIDAR-based virtual map; and (5) generate anupdated LIDAR-based virtual map of the store displaying aisles of thestore and the overlaid locations of the individual goods within thestore. The computer system may include additional, less, or alternatefunctionality, including that discussed elsewhere herein.

For instance, the executable instructions may further cause the computersystem to: overlay customer flow through the store on top of the updatedLIDAR-based virtual map; and display the updated LIDAR-based virtual mapincluding the overlaid customer flow though the store on a mobile deviceof the customer or an employee.

The executable instructions may further cause the computer system to:receive a virtual or electronic customer list for items to be purchasedfrom an App installed on a customer mobile device via one or more radiofrequency links, and/or via wireless communication or data transmission.

The executable instructions may further cause the computer system to:receive confirmation that items of the virtual or electronic customerlist have been picked up or delivered; charge a virtual pay account forthe picked up or delivered items; and receive payment for the picked upor delivered items from the virtual pay account.

In another aspect, a computer-implemented method for commercialinventory mapping may be provided. The method may include, via one ormore local or remote processors, sensors, servers, LIDAR devices, and/ortransceivers: (1) receiving sensor data via wireless communication ordata transmission over one or more radio frequency links, the sensordata associated with item movement or purchase, the sensor data beinggenerated from a good-mounted sensor, shelf-mounted sensor, a camera, ora self-check device; (2) updating an electronic inventory of goodswithin a store based upon the received sensor data associated with itemmovement or purchase; (3) receiving an electronic order of goods from acustomer mobile device via wireless communication or data transmissionover one or more radio frequency links; (4) determining goods in theelectronic order received from the customer that are still available bycomparing the updated electronic inventory of goods with the electronicorder of goods and/or comparing the electronic order of goods with otherincoming electronic orders from other customers; (5) generating aLIDAR-based virtual map of the store from processor analysis of LIDARdata; (6) determining a location of goods in the electronic order thatare still available; (7) overlaying the determined location of the goodsto the LIDAR-based virtual map of the store; and/or (8) generating anupdated LIDAR-based virtual map of the store displaying aisles of thestore and the determined locations of the goods within the store. Themethod may include additional, less, or alternate actions, includingthose discussed elsewhere herein.

For instance, the updated LIDAR-based virtual map may also depict apre-determined flow through the store to maintain customer trafficuniform or spaced out.

The method may include, via the one or more local or remote processors,servers, transceivers, and/or sensors, receiving sensor data indicatingthat the customer has picked up a specific item or placed the specificitem in a cart, the sensor data being associated with the specific item;and/or updating an electronic or virtual inventory of the store basedupon the sensor data to indicate that the specific item has been pickedup by a customer, or placed in the cart.

The location of the goods in the electronic order may be determined fromprocessor analysis of the LIDAR data. The method further may furtherinclude, via the one or more local or remote processors, sensors,servers, and/or transceivers: in response to the determination that thegoods in the electronic order are still available, reserving, for thecustomer, the goods in the electronic order that are still available.

The method may further include, via the one or more local or remoteprocessors, sensors, servers, and/or transceivers: generating iconscorresponding to the goods in the electronic order are that are stillavailable; and overlaying the icons onto the LIDAR-based virtual map ofthe store.

The method may further include, via the one or more local or remoteprocessors, sensors, servers, and/or transceivers: receivingconfirmation that items of the virtual or electronic customer list havebeen picked up or delivered; charging a virtual pay account for thepicked up or delivered items; and receiving payment for the picked up ordelivered items from the virtual pay account.

In another aspect, a computer system configured for commercial inventorymapping may be provided. The computer system may include one or morelocal or remote processors, sensors, servers, LIDAR devices, and/ortransceivers configured to: (1) receive sensor data via wirelesscommunication or data transmission over one or more radio frequencylinks, the sensor data associated with item movement or purchase, thesensor data being generated from a good-mounted sensor, shelf-mountedsensor, a camera, or a self-check device; (2) update an electronicinventory of goods within a store based upon the received sensor dataassociated with the item movement or purchase; (3) receive an electronicorder of goods from a customer mobile device via wireless communicationor data transmission over one or more radio frequency links; (4)determine if the goods in the electronic order received from thecustomer are still available by comparing the updated electronicinventory of goods with the electronic order of goods and/or comparingthe electronic order of goods with other incoming electronic orders fromother customers; (5) generate a LIDAR-based virtual map of the storefrom processor analysis of LIDAR data; (6) determine a location of thegoods in the electronic order that are still available; (7) overlay thedetermined location of the individual goods onto the LIDAR-based virtualmap of the store; and/or (8) generate and display an updated LIDAR-basedvirtual map of the store displaying aisles of the store and thedetermined location of the goods within the store. The system may beconfigured with additional, less, or alternate functionality, includingthat discussed elsewhere herein.

For instance, the updated LIDAR-based virtual map may also depict apre-determined flow through the store for customers to follow, such asto facilitate social distancing.

The system and/or one or more local or remote processors, servers,transceivers, and/or servers may also be configured to: receive sensordata indicating that the customer has picked up a specific item orplaced the specific item in a cart via wireless communication or datatransmission over one or more radio frequency links, the sensor databeing associated with the specific item; and/or update an electronicinventory of items within the store based upon the sensor data toindicate that the item has been picked up by a customer, or placed inthe cart.

The location of the goods in the electronic order may be determined fromprocessor analysis of the LIDAR data.

The system may be further configured to: in response to thedetermination that the goods in the electronic order are stillavailable, reserve, for the customer, the goods in the electronic orderthat are still available.

The system may be further configured to: generate icons corresponding tothe goods in the electronic order are that are still available; andoverlay the icons onto the LIDAR-based virtual map of the store.

The system may be further configured to: receive confirmation that itemsof the virtual or electronic customer list have been picked up ordelivered; charge a virtual pay account for the picked up or delivereditems; and receive payment for the picked up or delivered items from thevirtual pay account.

In another aspect, a computer system configured for commercial inventorymapping may be provided. The computer system may include: one or moreprocessors; and a program memory coupled to the one or more processorsand storing executable instructions that when executed by the one ormore processors cause the computer system to: (1) receive sensor datavia wireless communication or data transmission over one or more radiofrequency links, the sensor data associated with item movement orpurchase, the sensor data being generated from a good-mounted sensor,shelf-mounted sensor, a camera, or a self-check device; (2) update anelectronic inventory of goods within a store based upon the receivedsensor data associated with the item movement or purchase; (3) receivean electronic order of goods from a customer mobile device via wirelesscommunication or data transmission over one or more radio frequencylinks; (4) determine if the goods in the electronic order received fromthe customer are still available by comparing the updated electronicinventory of goods with the electronic order of goods and/or comparingthe electronic order of goods with other incoming electronic orders fromother customers; (5) generate a LIDAR-based virtual map of the storefrom processor analysis of LIDAR data; (6) determine a location of thegoods in the electronic order that are still available; (7) overlay thedetermined location of the goods onto the LIDAR-based virtual map of thestore; and (8) generate and display an updated LIDAR-based virtual mapof the store displaying aisles of the store and the determined locationof the goods within the store. The computer system may includeadditional, less, or alternate functionality, including that discussedelsewhere herein.

For instance, the updated LIDAR-based virtual map may also depict apre-determined flow through the store for customers to follow.

The executable instructions may further cause the computer system to:receive sensor data indicating that the customer has picked up aspecific item or placed the specific item in a cart via wirelesscommunication or data transmission over one or more radio frequencylinks, the sensor data being associated with the specific item; andupdate an electronic inventory of items within the store based upon thesensor data to indicate that the item has been picked up by a customer,or placed in the cart.

The location of the goods in the electronic order may be determined fromprocessor analysis of the LIDAR data.

The executable instructions may further cause the computer system to: inresponse to the determination that the goods in the electronic order arestill available, reserve, for the customer, the goods in the electronicorder that are still available.

The executable instructions may further cause the computer system to:generate icons corresponding to the goods in the electronic order arethat are still available; and overlay the icons onto the LIDAR-basedvirtual map of the store.

In another aspect, a computer-implemented method for inventory mappingmay be provided. The method may include, via one or more local or remoteprocessors, sensors, servers, transceivers, and/or LIDAR devices, (1)receiving a customer virtual order form their mobile device; (2)verifying which items are still available for purchase (such as bycomparing the virtual order with an up-to-date inventory list and/orwith other customer orders arriving near simultaneously); (3) holding orreserving the items currently available for the customer and blockingother orders for the same items; (4) generating a LIDAR-based virtualmap of the store with the locations and/or aisles of each available itembeing graphically depicted (such as by an icon); (5) displaying theLIDAR-based map along with the good icons on the mobile device of thecustomer (or an employee, such as in the case of curb-side pickup);remotely verifying pickup of the desired goods by the customer (such asvia cameras, or good-mounted sensors or shelf-mounted sensors); and/orprocessing payment from a financial account of the customer. The methodmay include additional, less, or alternate actions, including thosediscussed elsewhere herein.

Exemplary System for 3D Generation of a Floor Plan for a CommercialBuilding

FIG. 15 shows an exemplary computer system for 3D generation of a floorplan for a commercial building. With reference thereto, each server 110may include one or more computer processors adapted and configured toexecute various software applications and components of the system for3D generation of a floor plan for a commercial building. In particular,the server 110 may include commercial building designer 1541 (located,e.g., on program memory 160).

In some embodiments, the commercial building designer 1541 comprises amachine learning algorithm for 3D generation of a floor plan for acommercial building. Exemplary types of buildings include officebuildings, stores, warehouses, etc. However, the machine learningalgorithm may provide 3D generation of a floor plan for a commercialbuilding of any kind.

The machine learning algorithm may be trained using preexisting data(e.g., from database 1530, or database 146, etc.). The preexisting datamay include general data of commercial building layouts, 3D models offloor plans, 3D models of floor plans of a specific company, data of howa floor plan effects a company's efficiency or profits, etc. In someembodiments, this preexisting data is gathered from websites or fromdatabases of individual companies. In some embodiments, the preexistingdata may be generated from various devices illustrated in FIG. 15 (e.g.,LIDAR camera 120, photographic camera 125, drone 140, smart device 710,etc.); for instance, the data gathered from these devices may be used tobuild 3D models of floor plans which are stored in database 1530.Furthermore, the machine learning algorithm may be trained based uponknowledge of correlations between floorplans and a company's efficiencyor profits.

Furthermore, the machine learning algorithm may create a profile for aparticular company. The profile may be based upon, for instance, floorplans in other buildings of the company. In this way, the system maygenerate a recommended floor plan in a style that may be preferred bythe company. The profile may be created as part of the training phase.

The machine learning algorithm may be used to analyze any or all of thedata held by servers 110 and/or database 1530. The machine learningalgorithm may be a supervised learning algorithm, employ decision trees,make use of an artificial neural network, make use of Bayesianstatistical analysis, or combinations thereof. In this regard, aprocessor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning algorithm mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, or a combined learning module or programthat learns in two or more fields or areas of interest. Machine learningmay involve identifying and recognizing patterns in existing data inorder to facilitate making predictions for subsequent data. Models maybe created based upon example inputs in order to make valid and reliablepredictions for novel inputs. In some embodiments, the machine learningalgorithm is a convolutional neural network (CNN); in some embodiments,the machine learning algorithm is a deep neural network (DNN); in someembodiments, the machine learning algorithm is a recurrent neuralnetwork (RNN). It is advantageous to use machine learning algorithmsthat are particularly adept at finding patterns (e.g., CNNs).

Although the exemplary system 1500 is illustrated to include one LIDARcamera 120, one drone 140, and one group of servers 110 (FIG. 15 isillustrated to show three servers 110, but it should be understood thatthe server(s) 110 may be one or more server(s)), it should be understoodthat different numbers LIDAR camera 120, drone 140, servers 110, and/orany of the other elements shown in FIG. 15 may be utilized. Forinstance, the system 1500 may include any number of servers 110 andhundreds of mobile LIDAR cameras 120 or drones 140. Furthermore, thedatabase storage or processing performed by the one or more servers 110may be distributed among a plurality of servers 110 in an arrangementknown as “cloud computing.” This configuration may provide variousadvantages, such as enabling near real-time uploads and downloads ofinformation as well as periodic uploads and downloads of information.

The server 110 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For instance, separate databases may be used for storingdifferent types of information and/or making different calculations. Thecontroller 155 may include a program memory 160, a processor 162 (whichmay be called a microcontroller or a microprocessor), a random-accessmemory (RAM) 164, and an input/output (I/O) circuit 166, all of whichmay be interconnected via an address/data bus 165. It should beappreciated that although only one microprocessor 162 is shown, thecontroller 155 may include multiple microprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a database data receivingapplication 1543 for receiving data from a database such as database1530; camera data receiving application 144 for receiving camera data; a3D model builder/modifier 145 for building and modifying a 3D model(e.g., of a commercial building with a commercial inventory etc.); adisplay application 146 for displaying a representation of the 3D model;a navigation input receiver 147 for receiving navigation input; acommercial building designer 1541 for generating a design for acommercial building, such as a floor plan of a commercial building;and/or a directions providing application 1545 for providing directions,such as directions to an object in a commercial building. The varioussoftware applications may be executed on the same computer processor oron different computer processors.

Furthermore, in the exemplary system of FIG. 15 , the servers 110 areshown sending and receiving information with LIDAR camera 120. Thegathered LIDAR data may be analyzed to determine the dimensions of anobject (e.g., length, width, height, curvature, etc.) and/or theinterior of a commercial building.

The LIDAR data may also be used to determine interior and/or exteriordimensions of the commercial building. For instance, interior dimensionsof hallways, offices, meeting rooms, etc. may be measured by the LIDARdata. The LIDAR data may be used to create a partial or complete floorplan and/or 3D model of the commercial building. Furthermore, 3D pointcloud(s) may be created from the LIDAR data. The LIDAR camera 120 may beoperated by any human or machine.

In some embodiments, the LIDAR camera 120 may be operated by the ownerof a company and/or commercial building. For instance, an individual whoowns both a company and a commercial building that the company operatesin may wish to improve the company's efficiency by rearranging officespace in the commercial building. In accordance with the techniquesdescribed herein, the individual may operate LIDAR camera 120 to createa 3D model of a floor of the commercial building. The system, using atrained machine learning algorithm, may then provide suggestions abouthow to rearrange the office space on the floor to improve efficiencyand/or profits.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the commercial building and/or company. For instance,data may be gathered from public records, property deeds, governmentrecords, websites, previous insurance claims, and so forth. In someembodiments, this data is gathered from database 1530.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Exemplary Computer-Implemented Method for 3D Generation of a Floor Planfor a Commercial Building

FIG. 16 shows a flowchart of an exemplary computer-implemented method1600 for viewing potential placement of an object in a room. Withreference thereto, at optional step 1602, as described above, the systemmay train a machine learning algorithm, such as a machine learningalgorithm comprised on commercial building designer 1541.

At step 1604, LIDAR data (e.g., from LIDAR camera 120) is received, suchas via wireless communication or data transmission over one or moreradio frequency links, by server 110. In some embodiments, the LIDARdata is transferred to the servers 110 from the LIDAR camera 120 via aphysical memory device.

At optional step 1606, the server 110 receives any of: camera data fromcamera 125; smart device data from smart devices 710; data from database1530; and/or drone data from drone 140.

At step 1608, the system builds (e.g., with 3D model builder/modifier145) a 3D model of a commercial building and/or floor of the commercialbuilding based upon: the LIDAR data; the camera data; the commercialbuilding data; the smart device data, and/or the drone data.

At step 1610, the commercial building designer 1541 receives the 3Dmodel of a commercial building and/or floor of the commercial building(e.g., from the 3D model builder/modifier 145, or from another sourcesuch as database 1530 or database 146).

At optional step 1612, input from a human user is received regarding anew floor plan. For instance, a user may know that she will place aparticular office (or any other kind of room such as a meeting room,kitchen, etc.) in a particular location on a floor; the user may theninput the location of this office, and the system will then create thenew floor plan based upon the input location of this office. In otherwords, the user may input the first part of a floor plan, and the systemwill fill in the rest.

At step 1614, using the (trained) machine learning algorithm, a newfloor plan of a floor of the commercial building is generated. The newfloor plan may be based upon the received 3D model(s), and optionallyfurther based upon other criteria such as the input from the human userreceived at step 1612. The new floor plan may comprise changes to anexisting floor plan that improve company efficiency and/or profits(e.g., a change to office locations, or a change to a layout of ameeting room).

At step 1616, a representation of the new floor plan is displayed.

At optional step 1618, dimensional data may be displayed. For instance,dimensional data of room(s) in the floor plan may be overlaid onto thedisplayed representation of the new floor plan. Additionally oralternatively, dimensional data of an object (e.g., table, desk,electronic equipment, etc.) may be displayed. In some embodiments, thedimensional data may be determined from any of the LIDAR data, thecamera data, the smart device data, the database data (e.g., fromdatabase 1530) and/or the drone data.

At optional step 1620, a 2D image (e.g., a snapshot from a 3D floorplan) is displayed. The 2D image may be displayed based upon a commandfrom a human user. The displayed 2D image may include dimensional data,such as the dimensions mentioned above with respect to step 1618.

At optional step 1622, the system generates a recommendation forplacement of an object in a room of the floorplan. In some embodiments,as mentioned above, the recommendation may be provided by a machinelearning algorithm of commercial building designer 1541.

At optional step 1624, a representation of an object in the floor planis moved according to a command from a human user. For instance, a usermay be considering where to place electronic equipment, such as amonitor, in a conference room, and may use this option to consider orvisualize different placements of the electronic equipment.

Exemplary System for 3D Navigation of an Interior of a Building

FIG. 17 shows an exemplary computer system for navigation of an interiorof a building. With reference thereto, each server 110 may include oneor more computer processors adapted and configured to execute varioussoftware applications and components of the system for 3D navigation ofan interior of a building.

In some embodiments, the system receives a 3D model of a building. Thebuilding may be any kind of building (e.g., an office building, agrocery or any other kind of store, a warehouse, etc.). In someembodiments, the system provides, to a user, navigation instructions fornavigating to a room of the building, such as an office, a conferenceroom, a kitchen, or a refrigeration room.

In some embodiments, the system provides navigation instructions to acommercial item in the building, such as a grocery item (e.g., eggs,milk, yogurt, bread, a particular kind of spice, etc.). In someembodiments, a user inputs a list of commercial items (e.g., a grocerylist), and the system determines a most efficient (e.g., a mostdistance-efficient, or a most time-efficient route) to obtain all of theitems on the list.

With reference to FIG. 15 , the server 110 may have a controller 155that is operatively connected to the database 146 via a link 156. Itshould be noted that, while not shown, additional databases may belinked to the controller 155 in a known manner. For instance, separatedatabases may be used for storing different types of information and/ormaking different calculations. The controller 155 may include a programmemory 160, a processor 162 (which may be called a microcontroller or amicroprocessor), a random-access memory (RAM) 164, and an input/output(I/O) circuit 166, all of which may be interconnected via anaddress/data bus 165. It should be appreciated that although only onemicroprocessor 162 is shown, the controller 155 may include multiplemicroprocessors 162.

Similarly, the memory of the controller 155 may include multiple RAMs164 and multiple program memories 160. Although the I/O circuit 166 isshown as a single block, it should be appreciated that the I/O circuit166 may include a number of different types of I/O circuits. The RAM 164and program memories 160 may be implemented as semiconductor memories,magnetically readable memories, or optically readable memories, forinstance.

The server 110 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 110 may include: a LIDAR data monitoring application 141 forreceiving information from LIDAR camera 120; a drone data monitoringapplication 142 for monitoring drone data; a database data receivingapplication 1743 for receiving data from a database such as database1730; camera data receiving application 144 for receiving camera data; a3D model builder/modifier 145 for building and modifying a 3D model of abuilding; a display application 146 for displaying a representation ofthe 3D model; a navigation input receiver 147 for receiving navigationinput; a navigation calculator 1741 for calculating navigationinstructions; and/or a directions providing application 1745 forproviding directions. The various software applications may be executedon the same computer processor or on different computer processors.

Furthermore, in the exemplary system of FIG. 17 , the servers 110 areshown sending and receiving information with LIDAR camera 120. Thegathered LIDAR data may be analyzed to determine the interior orexterior dimensions of a building and/or object(s) in the building. Forinstance, interior dimensions of hallways, offices, meeting rooms, storeaisles, etc. may be measured by the LIDAR data. The LIDAR data may beused to create a partial or complete floor plan and/or 3D model of thecommercial building. Furthermore, 3D point cloud(s) may be created fromthe LIDAR data. The LIDAR camera 120 may be operated by any human ormachine.

The LIDAR data may be sent to the servers 110 by any method. Forinstance, the LIDAR data may be sent to the servers 110 directly fromthe LIDAR camera 120 via the internet. In another example, the LIDARdata may be transferred from the LIDAR camera 120 to a computer (via,e.g., a cable, a USB device, or any other means), and then sent from thecomputer to the servers 110 by any methods (e.g., sent by the internet,by Ethernet connection, or so forth).

The servers 110 also gather data from other sources. For instance, theservers 110 may gather data from photographic camera 125 (e.g., anoptical instrument used for photography or to record images, etc.). Insome embodiments, the camera 125 is a camera on an individual'ssmartphone. The camera data gathered by camera 125 includes color data,pixel data, and so forth.

Moreover, the servers 110 may also gather data from preexisting sourcesthat have data on the building or objects in the building. For instance,data may be gathered from a company's own records, public records,property deeds, government records, websites, previous insurance claims,and so forth. In some embodiments, this data is gathered from database1730 and/or database 146.

The servers 110 also gather data from a drone 140. Such data may includedata from a camera on the drone, a LIDAR camera on the drone, radiodetection and ranging (RADAR) data gathered by the drone, globalpositioning system (GPS) data gathered by the drone, information from aninfrared camera of the drone, and so forth.

Although the exemplary computer system 1700 is illustrated to includeone LIDAR camera 120, one drone 140, and one group of servers 110 (FIG.17 is illustrated to show three servers 110, but it should be understoodthat the server(s) 110 may be one or more server(s)), it should beunderstood that different numbers LIDAR camera 120, drone 140, servers110, and/or any of the other elements shown in FIG. 17 may be utilized.For instance, the system 1700 may include any number of servers 110 andhundreds of mobile LIDAR cameras 120 or drones 140. Furthermore, thedatabase storage or processing performed by the one or more servers 110may be distributed among a plurality of servers 110 in an arrangementknown as “cloud computing.” This configuration may provide variousadvantages, such as enabling near real-time uploads and downloads ofinformation as well as periodic uploads and downloads of information.

A machine learning algorithm may be used to analyze any or all of thedata held by servers 110 and/or database 1730. The machine learningalgorithm may be a supervised learning algorithm, employ decision trees,make use of an artificial neural network, make use of Bayesianstatistical analysis, or combinations thereof. In this regard, aprocessor or a processing element may be trained using supervised orunsupervised machine learning, and the machine learning algorithm mayemploy a neural network, which may be a convolutional neural network, adeep learning neural network, or a combined learning module or programthat learns in two or more fields or areas of interest. Machine learningmay involve identifying and recognizing patterns in existing data inorder to facilitate making predictions for subsequent data. Models maybe created based upon example inputs in order to make valid and reliablepredictions for novel inputs. In some embodiments, the machine learningalgorithm is a convolutional neural network (CNN); in some embodiments,the machine learning algorithm is a deep neural network (DNN); in someembodiments, the machine learning algorithm is a recurrent neuralnetwork (RNN). It is advantageous to use machine learning algorithmsthat are particularly adept at finding patterns (e.g., CNNs).

The machine learning algorithm may be trained using preexisting data(e.g., from database 1730, or database 146, etc.). The preexisting datamay include general data of a company's inventory, commercial buildinglayouts, 3D models of floor plans, 3D models of floor plans of aspecific company, data of how a floor plan effects a company'sefficiency or profits, etc. In some embodiments, this preexisting datais gathered from websites or from databases of individual companies. Insome embodiments, the preexisting data may be generated from variousdevices illustrated in FIG. 17 (e.g., LIDAR camera 120, photographiccamera 125, drone 140, smart device 710, etc.); for instance, the datagathered from these devices may be used to build 3D models of floorplans which are stored in database 1730.

Exemplary Computer-Implemented Method for 3D Navigation of an Interiorof a Building

FIG. 18 shows a flowchart of an exemplary computer-implemented method1800 for viewing potential placement of an object in a room. Withreference thereto, at optional step 1802, the system may train a machinelearning algorithm, such as a machine learning algorithm used todetermine that a user is having difficulty locating a room and/or anitem, and as will be described below with respect to step 1818.

At step 1804, LIDAR data (e.g., from LIDAR camera 120) is received, suchas via wireless communication or data transmission over one or moreradio frequency links, by server 110. In some embodiments, the LIDARdata is transferred to the servers 110 from the LIDAR camera 120 via aphysical memory device.

At optional step 1806, the server 110 receives any of: camera data fromcamera 125; building data (including, e.g., dimensions of the building,a 3D model of the building, etc.), inventory data (e.g., a list ofcommercial items such as groceries along with their locations in thebuilding, etc.), smart device data from smart devices 710; data fromdatabase 1730; and/or drone data from drone 140.

At step 1808, the system builds (e.g., with 3D model builder/modifier145) a 3D model of a commercial building and/or floor of the commercialbuilding based upon: the LIDAR data, and/or any of the data received instep 1806. The 3D model includes: (i) a plurality of dimensions of theinterior of the building, and/or (ii) a location of a room and/or alocation of a commercial item. In some embodiments, the room is anoffice, a conference room, a kitchen, or a refrigeration room. In someembodiments, the commercial item is a grocery item, a medical item, afurniture item, or an electronics item.

At step 1810, the navigation calculator 1741 receives the 3D model ofthe building (e.g., from the 3D model builder/modifier 145, or fromanother source, such as database 1730 or database 146). In someembodiments, the 3D model is simply received from the building data heldin database 1730 or database 146, and thus does not need to be built bythe 3D model builder/modifier 145.

At step 1812, a request is received from a user to provide navigationinstructions to the room and/or the commercial item. In someembodiments, the request includes a list of commercial items, such as agrocery list.

At step 1814, the navigation instructions are calculated based upon thereceived 3D model of the building. In some embodiments, the navigationinstructions include a height level of the commercial item. Forinstance, a grocery item may be indicated as being on the third shelffrom the ground, or as a certain geometric distance from the ground. Insome embodiments, the navigation instructions also include an aislenumber that the commercial item is located at.

At step 1816, the user is provided with the calculated navigationinstructions to the room and/or the commercial item. If the requestincludes a list of commercial items, the system may determine a mostefficient (e.g., a most distance-efficient, or a most time-efficientroute) to obtain all of the items on the list. In some embodiments, amost time-efficient route is calculated based upon: (i) the locations ofitems on the list, and/or (ii) how difficult it is to navigate to anitem due to obstructions, such as crowds (e.g., data from camera 125indicates that a particular aisle is crowded, which the system takesinto account when calculating the navigation instructions).

In some embodiments, the provided navigation instructions are overlaidonto a pair of computerized glasses.

At optional step 1818, the machine learning algorithm (optionallytrained in step 1802) determines if the user is having difficultylocating the room and/or commercial item. If so, the system provides aremedy in response to the determination. For instance, the system mayprovide the user with navigation instructions to a store employee.Additionally or alternatively, the system may alert the store employeethat the customer is having difficulty locating the item. Additionallyor alternatively, the system may provide alternative or more detailednavigation instructions to an item, such as providing a shelf level orheight off the ground that an item is located at.

The machine learning algorithm may be trained (e.g., at step 1802) basedupon any data, such as data held by database 1730, or database 146. Forinstance, the machine learning algorithm may be trained based upon 3Dmodels of buildings, navigation instructions to a room and/or commercialitem, data of how long it takes a user to find a room and/or commercialitem, etc.

The determination that the user is having difficulty locating the roomand/or commercial item may be added to a database along with otherinformation correlated with the determination (e.g., time spentsearching for the room/item, other information about the user, etc.).This information may then be used to determine (e.g., with a machinelearning algorithm) which rooms and/or items are difficult to locate.

Exemplary Functionality: 3D Home Model for Visualizing Proposed Changesto Home

In another aspect, computer-implemented method for visualizing proposedchanges to a home may be provided. The method may include, via one ormore processors, sensors, servers, and/or transceivers: (1) receivinglight detection and ranging (LIDAR) data generated from a LIDAR camera;(2) measuring a plurality of dimensions of a room of the home based uponprocessor analysis of the LIDAR data; (3) building a 3D model of theroom based upon the measured plurality of dimensions; (4) receiving anindication of a proposed change to the room; (5) modifying the 3D modelto include the proposed change to the room; and/or (6) displaying arepresentation of the modified 3D model. The proposed changed to theroom may comprise an addition or removal of a wall. Additionally oralternatively, the proposed changed to the room may comprise a remodelof a kitchen or bathroom. In some embodiments, the displaying of therepresentation of the modified 3D model comprises displaying a 2D imagegenerated from the 3D model. The method may include additional, less, oralternate actions, including those discussed elsewhere herein.

For instance, the method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers, receiving navigationinput; wherein the displaying the representation of the modified 3Dmodel comprises visually navigating through the 3D model based upon thereceived navigation input. The proposed changed to the room may comprisea repair to at least one of a window or a wall.

The building of the 3D model may further include building the 3D modelfurther based upon preexisting home structural data.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers, receiving camera data includingcolor data; and the building of the 3D model may further include:deriving dimensions of a wall based upon processor analysis of the LIDARdata; deriving a color of the wall based upon processor analysis of thecamera data; and/or filling, into the 3D model, the wall including thederived dimensions of the wall and the derived color of the wall.

The proposed changed to the room may include an addition or removal of askylight. The proposed change to a room may be received from a computingdevice of a human user, the computing device comprising: a computer, asmartphone, or a tablet.

In another aspect, a computer system configured to visualize proposedchanges to a home may be provided. The computer system may include oneor more processors, sensors, servers, and/or transceivers configured to:(1) receive light detection and ranging (LIDAR) data generated from aLIDAR camera; (2) measure a plurality of dimensions of a room of thehome based upon processor analysis of the LIDAR data; (3) build a 3Dmodel of the room based upon the measured plurality of dimensions; (4)receive an indication of a proposed change to the room; (5) modify the3D model to include the proposed change to the room; and/or (6) displaya representation of the modified 3D model. The proposed changed to theroom may include an addition or removal of a wall. The computer systemmay include additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the computer system may be further configured, via the oneor more processors, sensors, servers, and/or transceivers, to display a2D image generated from the 3D model. The system may further beconfigured, via the one or more processors, sensors, servers, and/ortransceivers, to: receive navigation input; and display therepresentation of the modified 3D model by visually navigating throughthe 3D model based upon the received navigation input.

The system may further be configured, via the one or more processors,sensors, servers, and/or transceivers, to: receive camera data includingcolor data; and build the 3D model by: deriving dimensions of a wallbased upon processor analysis of the LIDAR data; deriving a color of thewall based upon processor analysis of the camera data; and filling, intothe 3D model, the wall including the derived dimensions of the wall andthe derived color of the wall.

In yet another aspect, a computer system configured to visualizeproposed changes to a home may be provided. The computer system mayinclude: one or more processors; and a program memory coupled to the oneor more processors and storing executable instructions that whenexecuted by the one or more processors cause the computer system to:receive light detection and ranging (LIDAR) data generated from a LIDARcamera; measure a plurality of dimensions of a room of the home basedupon processor analysis of the LIDAR data; build a 3D model of the roombased upon the measured plurality of dimensions; receive an indicationof a proposed change to the room; modify the 3D model to include theproposed change to the room; and/or display a representation of themodified 3D model. The proposed changed to the room may be an additionor removal of a wall. The computer system may include additional, less,or alternate functionality, including that discussed elsewhere herein.

For instance, the executable instructions further cause the computersystem to display a 2D image generated from the 3D model. The executableinstructions may further cause the computer system to: receivenavigation input; and display the representation of the modified 3Dmodel by visually navigating through the 3D model based upon thereceived navigation input.

The executable instructions may further cause the computer system to:receive camera data including color data; and build the 3D model by:deriving dimensions of a wall based upon processor analysis of the LIDARdata; deriving a color of the wall based upon processor analysis of thecamera data; and filling, into the 3D model, the wall including thederived dimensions of the wall and the derived color of the wall.

Exemplary Functionality: 3D Home Model for Representation of Property

In another aspect, a computer-implemented method for representation of ahome may be provided. The method may include, via one or moreprocessors, sensors, servers, and/or transceivers: (1) receiving lightdetection and ranging (LIDAR) data generated from a LIDAR camera; (2)measuring a plurality of dimensions of the home based upon processoranalysis of the LIDAR data; (3) building a 3D model of the home basedupon the measured plurality of dimensions; and/or (4) displaying arepresentation of the 3D model by visually navigating through the 3Dmodel. The method may include additional, less, or alternate actions,including those discussed elsewhere herein.

For instance, the method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers, receiving navigationinput via wireless communication or data transmission over one or moreradio frequency links; wherein the visual navigation through the 3Dmodel is based upon the received navigation input.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: displaying an arrow on thedisplayed 3D model; and receiving navigation input via wirelesscommunication or data transmission over one or more radio frequencylinks, the navigation input comprising a user selection of the arrow;wherein the visual navigation through the 3D model is based upon thereceived navigation input.

The visual navigation through the 3D model may be based upon a defaultnavigation generated by a machine learning algorithm. The measuredplurality of dimensions may include a measured width of a wall; and thedisplaying the representation of the 3D model may further includeoverlaying a numerical value of the width of the wall onto a visualrepresentation of the wall in the displayed 3D model.

The measured plurality of dimensions may include a measured height andwidth of an object; and the displaying the representation of the 3Dmodel may further include overlaying numerical values of the height andwidth of the object onto a visual representation of the object in thedisplayed 3D model.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving, via wirelesscommunication or data transmission over one or more radio frequencylinks, a user selection of an object displayed in the displayed 3Dmodel; and displaying a 2D image of the selected object includingdisplaying numerical values of a height and a width of the object.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving camera data includingcolor data; wherein the 3D model is built further based upon the colordata.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers, receiving camera data includingcolor data; wherein the building of the 3D model may further include:deriving dimensions of a wall based upon processor analysis of the LIDARdata; deriving a color of the wall based upon processor analysis of thecamera data; and filling, into the 3D model, the wall including thederived dimensions of the wall and the derived color of the wall.

In another aspect, a computer system configured for 3D representation ofa home may be provided. The computer system may include one or moreprocessors, sensors, servers, and/or transceivers configured to: (1)receive light detection and ranging (LIDAR) data generated from a LIDARcamera; (2) measure plurality of dimensions of the home based uponprocessor analysis of the LIDAR data; (3) build a 3D model of the homebased upon the measured plurality of dimensions; and/or (4) display arepresentation of the 3D model by visually navigating through the 3Dmodel. The computer system may include additional, less, or alternatefunctionality, including that discussed elsewhere herein.

For instance, the computer system may be further configured to, via theone or more processors, sensors, servers, and/or transceivers: displayan arrow on the displayed 3D model; receive navigation input viawireless communication or data transmission over one or more radiofrequency links, the navigation input comprising a user selection of thearrow; and base the visual navigation through the 3D model upon thereceived navigation input.

The measured plurality of dimensions may include a measured width of awall; and the computer system may be further configured to, via the oneor more processors, sensors, servers, and/or transceivers, display therepresentation of the 3D model by overlaying a numerical value of thewidth of the wall onto a visual representation of the wall in thedisplayed 3D model.

The computer system may be, further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive, via wirelesscommunication or data transmission over one or more radio frequencylinks, a user selection of an object displayed in the displayed 3Dmodel; and display a 2D image of the selected object includingdisplaying numerical values of a height and a width of the object.

In yet another aspect, a computer system configured for 3Drepresentation of a home may be provided. The computer system mayinclude: one or more processors; and a program memory coupled to the oneor more processors and storing executable instructions that whenexecuted by the one or more processors cause the computer system to:receive light detection and ranging (LIDAR) data generated from a LIDARcamera; measure plurality of dimensions of the home based upon processoranalysis of the LIDAR data; build a 3D model of the home based upon themeasured plurality of dimensions; and/or display a representation of the3D model by visually navigating through the 3D model. The computersystem may include additional, less, or alternate functionality,including that discussed elsewhere herein.

For instance, the executable instructions may further cause the computersystem to receive navigation input via wireless communication or datatransmission over one or more radio frequency links; and the visualnavigation through the 3D model is based upon the received navigationinput.

The executable instructions may further cause the computer system to:display an arrow on the displayed 3D model; receive navigation input viawireless communication or data transmission over one or more radiofrequency links, the navigation input comprising a user selection of thearrow; and base the visual navigation through the 3D model upon thereceived navigation input.

The measured plurality of dimensions may include a measured width of awall; and the executable instructions may further cause the computersystem to display the representation of the 3D model by overlaying anumerical value of the width of the wall onto a visual representation ofthe wall in the displayed 3D model.

The measured plurality of dimensions may include a measured height andwidth of an object; and the executable instructions may further causethe computer system to display the representation of the 3D model byoverlaying numerical values of the height and width of the object onto avisual representation of the object in the displayed 3D model.

The executable instructions may further cause the computer system to:receive, via wireless communication or data transmission over one ormore radio frequency links, a user selection of an object displayed inthe displayed 3D model; and display a 2D image of the selected objectincluding displaying numerical values of a height and a width of theobject.

The executable instructions may further cause the computer system to:receive camera data including color data; and build the 3D model furtherbased upon the color data.

Exemplary Functionality: 3D Model for Viewing Potential Placement of anObject

In another aspect, a computer-implemented method for viewing potentialplacement of an object may be provided. The method may include, via oneor more processors, sensors, servers, and/or transceivers: (1) receivinglight detection and ranging (LIDAR) data generated from a LIDAR camera;(2) measuring a plurality of dimensions of the object based uponprocessor analysis of the LIDAR data; (3) receiving or generating a 3Dmodel of a room, the 3D model of the room including dimensional data ofthe room; (4) inserting a representation of the object into the 3D modelof the room based upon processor analysis of: (i) the plurality ofdimensions of the object measured from the LIDAR data; and (ii) thedimensional data of the room; and/or (5) displaying the 3D model of theroom with the inserted representation of the object. The method mayinclude additional, less, or alternate actions, including thosediscussed elsewhere herein.

The object may be a piece of furniture comprising one of: (1) a chair;(2) a couch; (3) a table; (4) a desk; and/or (5) a lamp. The object maybe an appliance comprising one of: (1) a refrigerator; (2) a stove; (3)a microwave; (4) a dishwasher; (5) an air fryer; (6) a laundry machine;or (7) a dryer.

The displaying of the 3D model of the room with the insertedrepresentation of the object may include displaying the 3D model of theroom with the inserted representation of the object on a pair ofglasses.

In some embodiments, the object may be a first object, and the LIDARdata may be first LIDAR data; and the computer-implemented may furtherinclude, via the one or more processors, transceivers, sensors, and/orservers: receiving second LIDAR data generated from the LIDAR camera;measuring a plurality of dimensions of a second object based uponprocessor analysis of the second LIDAR data; and replacing therepresentation of the first object with a representation of the secondobject in the 3D model of the room.

The displaying of the 3D model of the room with the inserted object mayinclude displaying a price of the object.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: determining that it is notpossible to bring the object into the room based upon processor analysisof: (i) dimensional data of a door of the room, and (ii) the pluralityof dimensions of the object measured from the LIDAR data; wherein thedisplaying of the 3D model of the room with the inserted representationof the object further comprises displaying a warning indicating that itis not possible to bring the object into the room.

The object may be an object of a plurality of objects, and thecomputer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: measuring dimensionaldata of each object of the plurality of objects based upon processoranalysis of received LIDAR data; presenting, to a user, a list ofobjects of the plurality of objects, the list including a price of eachobject of the plurality of objects; receiving, from the user, aselection from the list of objects; and displaying the 3D model of theroom with a representation of the selected object.

The measured plurality of dimensions may include a measured height andwidth of an object; and the displaying the representation of the 3Dmodel may further include overlaying numerical values of the height andwidth of the object onto the representation of the object in thedisplayed 3D model.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving camera data includingcolor data and transparency data of the object; wherein therepresentation of the object is built based upon the color data and thetransparency data of the object.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: changing a placement of theinserted representation of the object in the 3D model based upon acommand from a user.

In another aspect, a computer system configured to display a potentialplacement of an object may be provided. The computer system may includeone or more processors, sensors, servers, and/or transceivers configuredto: (1) receive light detection and ranging (LIDAR) data generated froma LIDAR camera; (2) measure a plurality of dimensions of the objectbased upon processor analysis of the LIDAR data; (3) receive or generatea 3D model of a room, the 3D model of the room including dimensionaldata of the room; (4) insert a representation of the object into the 3Dmodel of the room based upon processor analysis of: (i) the plurality ofdimensions of the object measured from the LIDAR data; and (ii) thedimensional data of the room; and/or (5) display the 3D model of theroom with the inserted representation of the object. The computer systemmay include additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the computer system may further be configured to, via theone or more processors, sensors, servers, and/or transceivers: display,on a pair of glasses, the 3D model of the room with the insertedrepresentation of the object.

The computer system may further be configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: display a price ofthe object when displaying the 3D model.

The object may be an object of a plurality of objects, and the computersystem may be further configured to, via the one or more processors,sensors, servers, and/or transceivers: measure dimensional data of eachobject of the plurality of objects based upon processor analysis ofreceived LIDAR data; present, to a user, a list of objects of theplurality of objects; receive, from the user, a selection from the listof objects; and display the 3D model of the room with a representationof the selected object.

The measured plurality of dimensions may include a measured height andwidth of an object; and the computer system may be further configuredto, via the one or more processors, sensors, servers, and/ortransceivers, display the representation of the 3D model by overlayingnumerical values of the height and width of the object onto therepresentation of the object in the displayed 3D model.

In another aspect, a computer system configured to display a potentialplacement of an object may be provided. The computer system may include:one or more processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera;measure a plurality of dimensions of the object based upon processoranalysis of the LIDAR data; receive or generate a 3D model of a room,the 3D model of the room including dimensional data of the room; inserta representation of the object into the 3D model of the room based uponprocessor analysis of: (i) the plurality of dimensions of the objectmeasured from the LIDAR data; and (ii) the dimensional data of the room;and/or display the 3D model of the room with the inserted representationof the object. The computer system may further include additional, less,or alternate functionality, including that discussed elsewhere herein.

For instance, the computer system may further include computerizedglasses; wherein the executable instructions further cause the computersystem to display the 3D model of the room with the insertedrepresentation of the object on the computerized glasses.

The executable instructions may further cause the computer system to:display a price of the object when displaying the 3D model.

The object may be an object of a plurality of objects, and theexecutable instructions may further cause the computer system to:measure dimensional data of each object of the plurality of objectsbased upon processor analysis of received LIDAR data; present, to auser, a list of objects of the plurality of objects; receive, from theuser, a selection from the list of objects; and display the 3D model ofthe room with a representation of the selected object.

Exemplary Functionality: AI Based Recommendations for Object Placementin a Home

In another aspect, a computer-implemented method for machine learningbased recommendation of object placement may be provided. The method mayinclude, via one or more processors, sensors, servers, and/ortransceivers: (1) training a machine learning algorithm based uponpreexisting data of object placement in a room; (2) receiving room datacomprising dimensional data of a room; (3) receiving object datacomprising: (i) dimensional data of an object; (ii) a type of theobject; and/or (iii) color data of the object; and/or (4) with thetrained machine learning algorithm, generating a recommendation forplacement of the object in the room based upon: (i) the received roomdata, and (ii) the received object data. The method may includeadditional, less, or alternate actions, including those discussedelsewhere herein.

The object may comprise a furniture piece comprising: a chair; a table;a desk; a couch; or a lamp. The recommendation may be a firstrecommendation, and the method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: with trained machinelearning algorithm, generating a second recommendation for placement ofthe object in the room; and presenting, as first and second options, thefirst and second recommendations to a user.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving an object placement inthe room from a user; and displaying both: (i) a representation of theobject placement in the room from the user, and (ii) a representation ofthe object placement generated by the machine learning algorithm,thereby allowing the user to compare the placements.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: displaying, on a display, thegenerated recommendation for placement of the object in the room.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving a placement of an itemfrom a user in the room; wherein the recommendation for object placementin the room is further based upon the received placement of the item.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: building a user profile basedupon furniture placement in a home of a user; wherein the recommendationfor object placement in the room is further based upon the user profile.

The object data may include all of: (i) the dimensional data of theobject; (ii) the type of the object; and (iii) the color data of theobject.

In some embodiments, the machine learning algorithm may be aconvolutional neural network.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving light detection andranging (LIDAR) data generated from a LIDAR camera; and measuring aplurality of dimensions of the object based upon processor analysis ofthe LIDAR data; wherein the object data comprises the dimensional dataof the object, and the dimensional data of the object comprises theplurality of dimensions of the object measured based upon the processoranalysis of the LIDAR data.

In another aspect a computer system configured for machine learningbased recommendation of object placement may be provided. The computersystem may include one or more processors, sensors, servers, and/ortransceivers configured to: (1) train a machine learning algorithm basedupon preexisting data of object placement in a room; (2) receive roomdata comprising dimensional data of a room; (3) receive object datacomprising: (i) dimensional data of an object; (ii) a type of theobject; and/or (iii) color data of the object; and/or (4) with thetrained machine learning algorithm, generate a recommendation forplacement of the object in the room based upon: (i) the received roomdata, and (ii) the received object data. The computer system may includeadditional, less, or alternate functionality, including that discussedelsewhere herein.

The recommendation may be a first recommendation, and the system may befurther configured to, via the one or more processors, sensors, servers,and/or transceivers: with trained machine learning algorithm, generate asecond recommendation for placement of the object in the room; andpresent, as first and second options, the first and secondrecommendations to a user.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive an objectplacement in the room from a user; and display both: (i) arepresentation of the object placement in the room from the user, and(ii) a representation of the object placement generated by the machinelearning algorithm, thereby allowing the user to compare the placements.

The computer system may further include: a display; wherein the computersystem is further configured to, via the one or more processors,sensors, servers, and/or transceivers: display, on the display, thegenerated recommendation for placement of the object in the room.

The type of the object may comprise: a chair; a table; a desk; a couch;a lamp; a bookshelf; a picture; or a painting.

In yet another aspect, a computer system configured for machine learningbased recommendation of object placement may be provided. The system mayinclude: one or more processors; and a program memory coupled to the oneor more processors and storing executable instructions that whenexecuted by the one or more processors cause the computer system to:train a machine learning algorithm based upon preexisting data of objectplacement in a room; receive room data comprising dimensional data of aroom; receive object data comprising: (i) dimensional data of an object;(ii) a type of the object; and/or (iii) color data of the object; and/orwith the trained machine learning algorithm, generate a recommendationfor placement of the object in the room based upon: (i) the receivedroom data, and (ii) the received object data. The computer system mayinclude additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

The recommendation may be a first recommendation, and wherein theexecutable instructions may further cause the computer system to: withtrained machine learning algorithm, generate a second recommendation forplacement of the object in the room; and present, as first and secondoptions, the first and second recommendations to a user.

The executable instructions may further cause the computer system to:receive an object placement in the room from a user; and display both:(i) a representation of the object placement in the room from the user,and (ii) a representation of the object placement generated by themachine learning algorithm.

The executable instructions may further cause the computer system to:build a user profile based upon furniture placement in a home of a user;wherein the recommendation for object placement in the room is furtherbased upon the user profile.

The room data may further include color data of the room, and a windowplacement in the room.

Exemplary Functionality: 3D Model for Visualization of Landscape Design

In another aspect, a computer-implemented method for visualization oflandscape design may be provided. The method may include, via one ormore processors, sensors, servers, and/or transceivers: (1) receivinglight detection and ranging (LIDAR) data generated from a LIDAR camera;(2) measuring a plurality of dimensions of a landscape based uponprocessor analysis of the LIDAR data; (3) building a 3D model of thelandscape based upon the measured plurality of dimensions, the 3D modelincluding: (i) a structure, and (ii) a vegetation; and/or (4) displayinga representation of the 3D model. The method may include additional,less, or alternate actions, including those discussed elsewhere herein.

The structure may comprise: a patio; a shed; a garage; a fence; or anoutside of a room of a house. The vegetation may comprise: a tree; aplant; or a flower. The 3D model may further include a pathway. Thestructure and the vegetation may be determined based upon processoranalysis of the LIDAR data.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving drone data; wherein the3D model of the landscape is built further based upon the received dronedata.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving global positioningsystem (GPS) data; wherein the 3D model of the landscape is builtfurther based upon the received GPS data. The displayed representationof the 3D model may comprise a 2D image of the landscape.

In some embodiments, the LIDAR camera is positioned on the ground; andthe method further comprises, via the one or more processors,transceivers, sensors, and/or servers, receiving drone data from adrone, the drone data comprising: (i) radio detection and ranging(RADAR) data gathered by the drone, and (ii) photographic camera datagathered by the drone; wherein the 3D model of the landscape is furtherbuilt based upon the received drone data.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving object data from auser; and inputting, into a machine learning algorithm: (i) data of the3D model of the landscape, and (ii) the received object data to generatea recommendation for placement of the object in the landscape; whereinthe object comprises one of: a patio; a shed; a garage; a fence; a tree;a plant; a flower; or a pathway.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving object data from auser; and inputting, into a machine learning algorithm: (i) data of the3D model of the landscape, and (ii) the received object data to generatea recommendation for placement of the object in the landscape; whereinthe machine learning algorithm comprises: a convolutional neural network(CNN); a deep neural network (DNN); or a recurrent neural network (RNN).

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: determining dimensional data ofboundaries of the landscape based upon: (i) preexisting property datafrom a database, and (ii) the LIDAR data; and overlaying the dimensionaldata of the boundaries onto the displayed representation of the 3Dmodel.

In another aspect, a computer system configured for visualization oflandscape design may be provided. The computer system may include one ormore processors, sensors, servers, and/or transceivers configured to:(1) receive light detection and ranging (LIDAR) data generated from aLIDAR camera; (2) measure a plurality of dimensions of a landscape basedupon processor analysis of the LIDAR data; (3) build a 3D model of thelandscape based upon the measured plurality of dimensions, the 3D modelincluding: (i) a structure, and (ii) a vegetation; and/or (4) display arepresentation of the 3D model. The computer system may includeadditional, less, or alternate functionality, including that discussedelsewhere herein.

The computer system may further be configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive drone data;and build the 3D model of the landscape further based upon the receiveddrone data.

The computer system may further be configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive object datafrom a user; and input, into a machine learning algorithm: (i) data ofthe 3D model of the landscape, and (ii) the received object data togenerate a recommendation for placement of the object in the landscape;wherein the machine learning algorithm comprises: a convolutional neuralnetwork (CNN); a deep neural network (DNN); or a recurrent neuralnetwork (RNN).

The computer system may further be configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: determine dimensionaldata of boundaries of the landscape based upon: (i) preexisting propertydata from a database, and (ii) the LIDAR data; and overlay thedimensional data of the boundaries onto the displayed representation ofthe 3D model.

In yet another aspect, a computer system configured for visualization oflandscape design may be provided. The computer system may include: oneor more processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera;measure a plurality of dimensions of a landscape based upon processoranalysis of the LIDAR data; and/or build a 3D model of the landscapebased upon the measured plurality of dimensions, the 3D model including:(i) a structure, and (ii) a vegetation; and display a representation ofthe 3D model. The computer system may include additional, less, oralternate functionality, including that discussed elsewhere herein.

The executable instructions may further cause the computer system to:receive drone data; and build the 3D model of the landscape furtherbased upon the received drone data.

The executable instructions may further cause the computer system to:receive object data from a user; and input, into a machine learningalgorithm: (i) data of the 3D model of the landscape, and (ii) thereceived object data to generate a recommendation for placement of theobject in the landscape; wherein the machine learning algorithmcomprises: a convolutional neural network (CNN); a deep neural network(DNN); or a recurrent neural network (RNN).

The executable instructions may further cause the computer system to:determine dimensional data of boundaries of the landscape based upon:(i) preexisting property data from a database, and (ii) the LIDAR data;and overlay the dimensional data of the boundaries onto the displayedrepresentation of the 3D model.

Exemplary Functionality: Visualization of Utility Lines

In another aspect, a computer-implemented method for visualization of autility line may be provided. The method may include, via one or moreprocessors, sensors, servers, and/or transceivers: (1) receiving lightdetection and ranging (LIDAR) data generated from a LIDAR camera; (2)receiving preexisting utility line data; and/or (3) determining alocation of the utility line based upon: (i) the received LIDAR data,and (ii) the received preexisting utility line data. The method mayinclude additional, less, or alternate actions, including thosediscussed elsewhere herein.

For instance, the method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: analyzing the LIDARdata to determine a location of an object; and determining a location ofthe object in the preexisting utility line data; wherein thedetermination of the location of the utility line is made by matchingthe location of the object determined from the LIDAR data with thelocation of the objected determined from the preexisting utility linedata.

The preexisting utility line data may include a geographic indication ofthe utility line. The utility line may comprise one of: a gas line; anelectric line; a water line; a cable line; or a fiber optic line. Thepreexisting utility line data may be received from a database of publicrecords.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving drone data; wherein thedetermination of the location of the utility line is based further uponthe drone data.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving drone data comprisingsecond LIDAR data and/or radio detection and ranging (RADAR); whereinthe determination of the location of the utility line is based furtherupon the drone data comprising the second LIDAR data and/or RADAR data.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving global positioningsystem (GPS) data of the LIDAR camera; wherein the determination of thelocation of the utility line is based further upon the GPS data.

The method may also include, via the one or more processors,transceivers, sensors, and/or servers: displaying, on a display, thelocation of the utility line. The method may further include, via theone or more processors, transceivers, sensors, and/or servers:providing, to a user, an indication of where to mark the ground for theutility line based upon the determined location of the utility line.

The method may further include, via the one or more processors,transceivers, sensors, and/or servers: providing, to a user, anindication of the location of the utility line by overlaying theindication of the location of the utility line onto an image or videodata of a ground.

The method may also include, via the one or more processors,transceivers, sensors, and/or servers: providing, to a user, anindication of the location of the utility line by overlaying theindication of the location of the utility line onto an image or video ofa ground; wherein the image or video is generated by a camera of asmartphone of the user.

The preexisting utility line data may be comprised in geographic data ofan area proximate to the utility line; the geographic data of the areaproximate to the utility line may further comprise data of a structurein the area proximate to the utility line; and the method may furtherinclude, via the one or more processors, transceivers, sensors, and/orservers: determining data of the structure based upon processor analysisof the LIDAR data, the data of the structure based upon processoranalysis of the LIDAR data comprising: (i) a location of a structure,and (ii) dimensional data of the structure; comparing (a) the data ofthe structure determined based upon the processor analysis of the LIDARdata with (b) the data of the structure comprised in the geographic dataof the area proximate to the utility line; determining, based upon thecomparison, that a location of the structure in the geographic data ofthe area proximate to the utility line is not correct; and displaying,on a display, an indication that the location of the structure in thegeographic data of the area proximate to the utility line is notcorrect.

In another aspect, a computer system configured for visualization of autility line may be provided. The computer system may include one ormore processors, sensors, servers, and/or transceivers configured to:(1) receive light detection and ranging (LIDAR) data generated from aLIDAR camera; (2) receive preexisting utility line data; and/or (3)determine a location of the utility line based upon: (i) the receivedLIDAR data, and (ii) the received preexisting utility line data.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive drone datacomprising second LIDAR data and/or radio detection and ranging (RADAR);wherein the determination of the location of the utility line is basedfurther upon the drone data comprising the second LIDAR data and/orRADAR data.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: provide, to a user,an indication of the location of the utility line by overlaying theindication of the location of the utility line onto an image or video ofa ground; wherein the image or video is generated by a camera of asmartphone of the user.

In yet another aspect, a computer system configured for visualization ofa utility line may be provided. The computer system may include: one ormore processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera;receive preexisting utility line data; and determine a location of theutility line based upon: (i) the received LIDAR data, and (ii) thereceived preexisting utility line data.

The executable instructions may further cause the computer system to:analyze the LIDAR data to determine a location of an object; anddetermine a location of the object in the preexisting utility line data;wherein the determination of the location of the utility line is made bymatching the location of the object determined from the LIDAR data withthe location of the objected determined from the preexisting utilityline data.

The executable instructions may further cause the computer system to:receive drone data comprising second LIDAR data and/or radio detectionand ranging (RADAR); wherein the determination of the location of theutility line is based further upon the drone data comprising the secondLIDAR data and/or RADAR data.

The executable instructions may further cause the computer system to:provide, to a user, an indication of the location of the utility line byoverlaying the indication of the location of the utility line onto animage or video of a ground; wherein the image or video is generated by acamera of a smartphone of the user.

Exemplary Functionality: Commercial Inventory Mapping

In another aspect, a computer-implemented method for commercialinventory mapping may be provided. The method may include, via one ormore processors, sensors, servers, and/or transceivers: (1) receivinglight detection and ranging (LIDAR) data generated from a LIDAR camera;(2) determining data of a first object based upon processor analysis ofthe LIDAR data, the data of the first object comprising: (i) dimensionaldata of the first object, and (ii) a type of the first object; and/or(3) adding the first object and the first object data to a commercialinventory list. The method may include additional, less, or alternateactions, including those discussed elsewhere herein.

For instance, the method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: determining data of asecond object based upon processor analysis of the LIDAR data, the dataof the second object comprising: (i) dimensional data of the secondobject, and (ii) a type of the second object; and adding the secondobject and the second object data to the commercial inventory list.

In some embodiments, the LIDAR data may be first LIDAR data, and themethod may further include, via the one or more processors,transceivers, sensors, and/or servers: receiving second LIDAR data;determining data of a second object based upon processor analysis of thesecond LIDAR data, the data of the second object comprising: (i)dimensional data of the second object, and (ii) a type of the secondobject; and adding the second object and the second object data to thecommercial inventory list.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: determining alocation of the first object based upon processor analysis of the LIDARdata; wherein the first object data further comprises the location ofthe first object.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: determining alocation of the first object based upon processor analysis of the LIDARdata; wherein the first object data further comprises the location ofthe first object; wherein the method further comprises, via the one ormore processors, transceivers, sensors, and/or servers: providing, to auser, directions to the first object based upon the determined locationof the first object.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: receiving globalpositioning system (GPS) data corresponding to the LIDAR data;determining a location of the first object based upon processor analysisof the LIDAR data and the corresponding GPS data; and wherein the firstobject data further comprises the location of the first object.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: receivingphotographic camera data from a photographic camera; wherein the type ofthe first object is determined further based upon processor analysis ofthe photographic camera data.

In some embodiments, the LIDAR data may be received from one or moredrones.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: receivingphotographic camera data from a photographic camera, the photographiccamera data comprising either image data or video data; wherein the typeof the first object is determined further based upon processor analysisrecognizing either a bar code or a Quick Response (QR) code in theeither image data or video data.

In another aspect, a computer system configured for commercial inventorymapping may be provided. The computer system may include one or moreprocessors, sensors, servers, and/or transceivers configured to: (1)receive light detection and ranging (LIDAR) data generated from a LIDARcamera; (2) determine data of a first object based upon processoranalysis of the LIDAR data, the data of the first object comprising: (i)dimensional data of the first object, and (ii) a type of the firstobject; and/or (3) add the first object and the first object data to acommercial inventory list.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: determine data of asecond object based upon processor analysis of the LIDAR data, the dataof the second object comprising: (i) dimensional data of the secondobject, and (ii) a type of the second object; and add the second objectand the second object data to the commercial inventory list.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: determine a locationof the first object based upon processor analysis of the LIDAR data;wherein the first object data further comprises the location of thefirst object.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: determine a locationof the first object based upon processor analysis of the LIDAR data;wherein the first object data further comprises the location of thefirst object; wherein the system is further configured to, via the oneor more processors, transceivers, sensors, and/or servers: provide, to auser, directions to the first object based upon the determined locationof the first object.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive globalpositioning system (GPS) data corresponding to the LIDAR data; anddetermine a location of the first object based upon processor analysisof the LIDAR data and the corresponding GPS data; wherein the firstobject data further comprises the location of the first object.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: determine if there isa second object in the LIDAR data; if there is a second object in theLIDAR data: determine data of the second object based upon processoranalysis of the LIDAR data, the data of the second object comprising:(i) dimensional data of the second object, and (ii) a type of the secondobject; and add the second object and the second object data to thecommercial inventory list; and if there is not a second object in theLIDAR data: receive additional LIDAR data generated from the LIDARcamera.

In yet another aspect, a computer system configured for commercialinventory mapping may be provided. The computer system may include: oneor more processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera;determine data of a first object based upon processor analysis of theLIDAR data, the data of the first object comprising: (i) dimensionaldata of the first object, and (ii) a type of the first object; and addthe first object and the first object data to a commercial inventorylist.

The executable instructions may further cause the computer system to:determine data of a second object based upon processor analysis of theLIDAR data, the data of the second object comprising: (i) dimensionaldata of the second object, and (ii) a type of the second object; and addthe second object and the second object data to the commercial inventorylist.

The executable instructions further cause the computer system to:determine a location of the first object based upon processor analysisof the LIDAR data; wherein the first object data further comprises thelocation of the first object.

The executable instructions may further cause the computer system to:determine a location of the first object based upon processor analysisof the LIDAR data; wherein the first object data further comprises thelocation of the first object; wherein the executable instructionsfurther cause the computer system to: provide, to a user, directions tothe first object based upon the determined location of the first object.

The executable instructions may further cause the computer system to:determine if there is a second object in the LIDAR data; if there is asecond object in the LIDAR data: determine data of the second objectbased upon processor analysis of the LIDAR data, the data of the secondobject comprising: (i) dimensional data of the second object, and (ii) atype of the second object; and add the second object and the secondobject data to the commercial inventory list; if there is not a secondobject in the LIDAR data: receive additional LIDAR data generated fromthe LIDAR camera.

Exemplary Functionality: 3D Generation of a Floor Plan for a CommercialBuilding

In another aspect, a computer-implemented method for 3D generation of afloor plan for a commercial building may be provided. The method mayinclude, via one or more processors, sensors, servers, and/ortransceivers: (1) receiving a 3-dimensional (3D) model of a floor of acommercial building comprising a plurality of dimensions of the floor ofthe commercial building; and (2) with a machine learning algorithm,generating a new floor plan of the floor of the commercial buildingbased upon the received 3D model of the floor; wherein the generated newfloor plan comprises a 3D floor plan. The method may include additional,less, or alternate actions, including those discussed elsewhere herein.

For instance, the computer-implemented method may further include, viathe one or more processors, transceivers, sensors, and/or servers:receiving light detection and ranging (LIDAR) data generated from aLIDAR camera; wherein the plurality of dimensions of the commercialbuilding are determined based upon processor analysis of the LIDAR data.

The new floor plan may include a 3D layout of a meeting room generatedby the machine learning algorithm. The computer-implemented method mayfurther include, via the one or more processors, transceivers, sensors,and/or servers: receiving, from a human user, an input of a first roomlocation on the floor; wherein the new floor plan is generated basedfurther upon the input first room location; and wherein the first roomcomprises: an office; a kitchen; a storage area; a meeting room; or aconference room.

The commercial building may comprise one or more offices. In someembodiments, the machine learning algorithm may be: a convolutionalneural network (CNN); a deep neural network (DNN); or a recurrent neuralnetwork (RNN).

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: prior to generatingthe new floor plan, training the machine learning algorithm based uponinput of a plurality of 3D models of floor plans.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: prior to generatingthe new floor plan, training the machine learning algorithm based uponinputs of: (i) a plurality of 3D models of floor plans, and (ii)correlations between: (a) individual 3D models of floor plans of theplurality of 3D models of floor plans, and (b) a company's efficiency orprofits.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: with the machinelearning algorithm, creating a company profile based upon 3D models offloor plans of the company; wherein the generation of the new floor planis based further upon the company profile.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: displaying arepresentation of the new floor plan on a display device includingoverlaying dimensional data of at least one room of the new floor planonto the representation of the new plan.

In another aspect, a computer system configured for 3D generation of afloor plan for a commercial building may be provided. The computersystem may include one or more processors, sensors, servers, and/ortransceivers configured to: (1) receive a 3-dimensional (3D) model of afloor of a commercial building comprising a plurality of dimensions ofthe floor of the commercial building; and (2) with a machine learningalgorithm, generate a new floor plan of the floor of the commercialbuilding based upon the received 3D model of the floor; wherein thegenerated new floor plan comprises a 3D floor plan. The computer systemmay include additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the computer system may be further configured to, via theone or more processors, sensors, servers, and/or transceivers: receivelight detection and ranging (LIDAR) data generated from a LIDAR camera;and determine the plurality of dimensions of the commercial buildingbased upon processor analysis of the LIDAR data.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: receive, from a humanuser, an input of a first room location on the floor; wherein the newfloor plan is generated based further upon the input first roomlocation; and wherein the first room comprises: an office; a kitchen; astorage area; a meeting room; or a conference room.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: prior to generatingthe new floor plan, train the machine learning algorithm based uponinput of a plurality of 3D models of floor plans.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: display arepresentation of the new floor plan on a display device by overlayingdimensional data of at least one room of the new floor plan onto therepresentation of the new plan.

In yet another aspect, a computer system configured for 3D generation ofa floor plan for a commercial building may be provided. The computersystem may include: one or more processors; and a program memory coupledto the one or more processors and storing executable instructions thatwhen executed by the one or more processors cause the computer systemto: receive a 3-dimensional (3D) model of a floor of a commercialbuilding comprising a plurality of dimensions of the floor of thecommercial building; and with a machine learning algorithm, generate anew floor plan of the floor of the commercial building based upon thereceived 3D model of the floor; wherein the generated new floor plancomprises a 3D floor plan. The computer system may include additional,less, or alternate functionality, including that discussed elsewhereherein.

For instance, the executable instructions further cause the computersystem to: receive light detection and ranging (LIDAR) data generatedfrom a LIDAR camera; and determine the plurality of dimensions of thecommercial building based upon processor analysis of the LIDAR data.

The executable instructions may further cause the computer system to:receive, from a human user, an input of a first room location on thefloor; wherein the new floor plan is generated based further upon theinput first room location; and wherein the first room comprises: anoffice; a kitchen; a storage area; a meeting room; or a conference room.

The executable instructions may further cause the computer system to:prior to generating the new floor plan, train the machine learningalgorithm based upon input of a plurality of 3D models of floor plans.

The executable instructions may further cause the computer system to:display a representation of the new floor plan on a display device byoverlaying dimensional data of at least one room of the new floor planonto the representation of the new plan.

Exemplary Functionality: 3D Navigation of an Interior of a Building

In another aspect, a computer-implemented method for 3D navigation of aninterior of a building may be provided. The method may include, via oneor more processors, sensors, servers, and/or transceivers: (1) receivinga 3-dimensional (3D) model of the building, the 3D model comprising: (i)a plurality of dimensions of the interior of the building, and (ii) alocation of a room and/or a location of a commercial item; (2)receiving, from a user, a request for navigation instructions to theroom and/or the commercial item; (3) calculating the navigationinstructions based upon the received 3D model of the building; and/or(4) providing, to the user, the calculated navigation instructions tothe room and/or the commercial item. The method may include additional,less, or alternate actions, including those discussed else-where herein.

For instance, the computer-implemented method may further include, viathe one or more processors, transceivers, sensors, and/or servers:receiving light detection and ranging (LIDAR) data generated from aLIDAR camera; wherein the plurality of dimensions of the interior of thebuilding are determined based upon processor analysis of the LIDAR data.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: receiving lightdetection and ranging (LIDAR) data generated from a LIDAR camera; basedupon processor analysis of the LIDAR data, determining: (i) the locationof the room in the interior of the building, and (ii) the location ofthe commercial item in the commercial building.

The location of the room may be included in the 3D model and thenavigation instructions; and the room may comprise: an office; aconference room; a kitchen; or a refrigeration room.

The location of the commercial item may be included in the 3D model andthe navigation instructions; and the commercial item may comprise: agrocery item; a medical item; a furniture item; or an electronics item.

The location of the commercial item may be included in the 3D model andthe navigation instructions; the commercial item may be a grocery item;and the navigation instructions may comprise: (i) an aisle number of thegrocery item, and (ii) a height level from the ground of the groceryitem.

The location of the commercial item may be included in the 3D model andthe navigation instructions; the request for navigation instructions maycomprise a list of commercial items including at least two commercialitems; and the provided navigation instructions may comprise a mostdistance-efficient route to obtain the at least two items of the list ofcommercial items.

The location of the commercial item may be included in the 3D model andthe navigation instructions; the request for navigation instructions maycomprise a list of commercial items including at least two commercialitems; and the provided navigation instructions may comprise a mosttime-efficient route to obtain the at least two items of the list ofcommercial items; and the most time-efficient route may be based upon:(i) the locations the at least two items of the commercial items, and(ii) camera data comprising customer density data.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: with a machinelearning algorithm, determining that the user is having difficultylocating the room and/or commercial item; and in response to thedetermination: (i) providing, to the user, navigation instructions to alocation of an employee of a store, and/or (ii) providing, to a companyemployee, an alert that the user is having difficulty locating the roomand/or commercial item.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: with a machinelearning algorithm, determining that the user is having difficultylocating the room and/or commercial item; and in response to thedetermination, providing, to the user, alternative navigationinstructions to the room and/or commercial item.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: adding, to adatabase, the provided navigation instructions along with other datareceived from the user; with a machine learning algorithm and based uponthe navigation instructions along with the other data received from theuser added to the database: diagnosing that a room and/or commercialitem is difficult to locate.

The computer-implemented method may further include, via the one or moreprocessors, transceivers, sensors, and/or servers: overlaying, onto apair of computerized glasses, the provided navigation instructions tothe room and/or commercial item.

In yet another aspect, a computer system configured for 3D navigation ofan interior of a building may be provided. The computer system mayinclude one or more processors, sensors, servers, and/or transceiversconfigured to: (1) receive a 3-dimensional (3D) model of the building,the 3D model comprising: (i) a plurality of dimensions of the interiorof the building, and (ii) a location of a room and/or a location of acommercial item; (2) receive, from a user, a request for navigationinstructions to the room and/or the commercial item; (3) calculate thenavigation instructions based upon the received 3D model of thebuilding; and/or (4) provide, to the user, the calculated navigationinstructions to the room and/or the commercial item. The computer systemmay include additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the computer system may be further configured to, via theone or more processors, sensors, servers, and/or transceivers: receivelight detection and ranging (LIDAR) data generated from a LIDAR camera;based upon processor analysis of the LIDAR data, determine: (i) thelocation of the room in the interior of the building, and (ii) thelocation of the commercial item in the commercial building.

The location of the commercial item may be included in the 3D model andthe navigation instructions; the request for navigation instructions maycomprise a list of commercial items including at least two commercialitems; and the provided navigation instructions comprise a mostdistance-efficient route to obtain the at least two items of the list ofcommercial items.

The computer system may be further configured to, via the one or moreprocessors, sensors, servers, and/or transceivers: add, to a database,the provided navigation instructions along with other data received fromthe user; and with a machine learning algorithm and based upon thenavigation instructions along with the other data received from the useradded to the database: diagnose that a room and/or commercial item isdifficult to locate.

In yet another aspect, a computer system configured for 3D navigation ofan interior of a building may be provided. The computer system mayinclude: one or more processors; and a program memory coupled to the oneor more processors and storing executable instructions that whenexecuted by the one or more processors cause the computer system to:receive a 3-dimensional (3D) model of the building, the 3D modelcomprising: (i) a plurality of dimensions of the interior of thebuilding, and (ii) a location of a room and/or a location of acommercial item; receive, from a user, a request for navigationinstructions to the room and/or the commercial item; calculate thenavigation instructions based upon the received 3D model of thebuilding; and/or provide, to the user, the calculated navigationinstructions to the room and/or the commercial item. The computer systemmay include additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

The executable instructions may further cause the computer system to:receive light detection and ranging (LIDAR) data generated from a LIDARcamera; based upon processor analysis of the LIDAR data, determine: (i)the location of the room in the interior of the building, and (ii) thelocation of the commercial item in the commercial building.

The location of the commercial item may be included in the 3D model andthe navigation instructions; the request for navigation instructions maycomprise a list of commercial items including at least two commercialitems; and the provided navigation instructions may comprise a mostdistance-efficient route to obtain the at least two items of the list ofcommercial items.

The executable instructions may further cause the computer system to:add, to a database, the provided navigation instructions along withother data received from the user; and with a machine learning algorithmand based upon the navigation instructions along with the other datareceived from the user added to the database: diagnose that a roomand/or commercial item is difficult to locate.

Other Matters

Although the text herein sets forth a detailed description of numerousdifferent embodiments, it should be understood that the legal scope ofthe invention is defined by the words of the claims set forth at the endof this patent. The detailed description is to be construed as exemplaryonly and does not describe every possible embodiment, as describingevery possible embodiment would be impractical, if not impossible. Onecould implement numerous alternate embodiments, using either currenttechnology or technology developed after the filing date of this patent,which would still fall within the scope of the claims.

It should also be understood that, unless a term is expressly defined inthis patent using the sentence “As used herein, the term ‘ ’ is herebydefined to mean . . . ” or a similar sentence, there is no intent tolimit the meaning of that term, either expressly or by implication,beyond its plain or ordinary meaning, and such term should not beinterpreted to be limited in scope based upon any statement made in anysection of this patent (other than the language of the claims). To theextent that any term recited in the claims at the end of this disclosureis referred to in this disclosure in a manner consistent with a singlemeaning, that is done for sake of clarity only so as to not confuse thereader, and it is not intended that such claim term be limited, byimplication or otherwise, to that single meaning. Finally, unless aclaim element is defined by reciting the word “means” and a functionwithout the recital of any structure, it is not intended that the scopeof any claim element be interpreted based upon the application of 35U.S.C. § 112(f).

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Additionally, certain embodiments are described herein as includinglogic or a number of routines, subroutines, applications, orinstructions. These may constitute either software (code embodied on anon-transitory, tangible machine-readable medium) or hardware. Inhardware, the routines, etc., are tangible units capable of performingcertain operations and may be configured or arranged in a certainmanner. In example embodiments, one or more computer systems (e.g., astandalone, client or server computer system) or one or more hardwaremodules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC) toperform certain operations). A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. Considering embodiments inwhich hardware modules are temporarily configured (e.g., programmed),each of the hardware modules need not be configured or instantiated atany one instance in time. For example, where the hardware modulescomprise a general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differenthardware modules at different times. Software may accordingly configurea processor, for example, to constitute a particular hardware module atone instance of time and to constitute a different hardware module at adifferent instance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multipleof such hardware modules exist contemporaneously, communications may beachieved through signal transmission (e.g., over appropriate circuitsand buses) that connect the hardware modules. In embodiments in whichmultiple hardware modules are configured or instantiated at differenttimes, communications between such hardware modules may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware modules have access. Forexample, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods or routines described herein may be at leastpartially processor-implemented. For example, at least some of theoperations of a method may be performed by one or more processors orprocessor-implemented hardware modules. The performance of certain ofthe operations may be distributed among the one or more processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment or as a server farm), while in other embodiments theprocessors may be distributed across a number of geographic locations.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment may be included in at leastone embodiment. The appearances of the phrase “in one embodiment” invarious places in the specification are not necessarily all referring tothe same embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the description. Thisdescription, and the claims that follow, should be read to include oneor at least one and the singular also includes the plural unless it isobvious that it is meant otherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs for theapproaches described herein. Thus, while particular embodiments andapplications have been illustrated and described, it is to be understoodthat the disclosed embodiments are not limited to the preciseconstruction and components disclosed herein. Various modifications,changes and variations, which will be apparent to those skilled in theart, may be made in the arrangement, operation and details of the methodand apparatus disclosed herein without departing from the spirit andscope defined in the appended claims.

The particular features, structures, or characteristics of any specificembodiment may be combined in any suitable manner and in any suitablecombination with one or more other embodiments, including the use ofselected features without corresponding use of other features. Inaddition, many modifications may be made to adapt a particularapplication, situation or material to the essential scope and spirit ofthe present invention. It is to be understood that other variations andmodifications of the embodiments of the present invention described andillustrated herein are possible in light of the teachings herein and areto be considered part of the spirit and scope of the present invention.

While the preferred embodiments of the invention have been described, itshould be understood that the invention is not so limited andmodifications may be made without departing from the invention. Thescope of the invention is defined by the appended claims, and alldevices that come within the meaning of the claims, either literally orby equivalence, are intended to be embraced therein.

It is therefore intended that the foregoing detailed description beregarded as illustrative rather than limiting, and that it be understoodthat it is the following claims, including all equivalents, that areintended to define the spirit and scope of this invention.

Furthermore, the patent claims at the end of this patent application arenot intended to be construed under 35 U.S.C. § 112(f) unless traditionalmeans-plus-function language is expressly recited, such as “means for”or “step for” language being explicitly recited in the claim(s). Thesystems and methods described herein are directed to an improvement tocomputer functionality, and improve the functioning of conventionalcomputers.

1. A computer-implemented method for visualizing proposed changes to ahome, the method comprising, via one or more processors, sensors,servers, and/or transceivers: receiving light detection and ranging(LIDAR) data generated from a LIDAR camera; measuring a plurality ofdimensions of a room of the home based upon processor analysis of theLIDAR data; building a 3D model of the room based upon the measuredplurality of dimensions; receiving an indication of a proposed change tothe room, wherein the proposed change to the room comprises a remodel ofa kitchen or bathroom; modifying the 3D model to include the proposedchange to the room; and displaying a representation of the modified 3Dmodel.
 2. The computer-implemented method of claim 1, wherein theproposed change to the room comprises addition or removal of a wall. 3.The computer-implemented method of claim 1, wherein the proposed changeto the room comprises a remodel of a kitchen, and includes a differentsize or color of a countertop or cabinet.
 4. The computer-implementedmethod of claim 1, wherein the displaying the representation of themodified 3D model comprises displaying a 2D image generated from the 3Dmodel.
 5. The computer-implemented method of claim 1, furthercomprising, via the one or more processors, transceivers, sensors,and/or servers, receiving navigation input; wherein the displaying therepresentation of the modified 3D model comprises visually navigatingthrough the 3D model based upon the received navigation input.
 6. Thecomputer-implemented method of claim 1, wherein the proposed change tothe room comprises a repair to at least one of a window or a wall. 7.The computer-implemented method of claim 1, wherein the building of the3D model further comprises building the 3D model further based uponpreexisting home structural data.
 8. The computer-implemented method ofclaim 1, further comprising, via the one or more processors,transceivers, sensors, and/or servers, receiving camera data includingcolor data; wherein the building of the 3D model further comprises:deriving dimensions of a wall based upon processor analysis of the LIDARdata; deriving a color of the wall based upon processor analysis of thecamera data; and filling, into the 3D model, the wall including thederived dimensions of the wall and the derived color of the wall.
 9. Thecomputer-implemented method of claim 1, wherein the proposed change tothe room comprises addition or removal of a skylight.
 10. Thecomputer-implemented method of claim 1, wherein the proposed change to aroom is received from a computing device of a human user, the computingdevice comprising: a computer, a smartphone, or a tablet.
 11. A computersystem configured to visualize proposed changes to a home, the computersystem comprising one or more processors, sensors, servers, and/ortransceivers configured to: receive light detection and ranging (LIDAR)data generated from a LIDAR camera; measure a plurality of dimensions ofa room of the home based upon processor analysis of the LIDAR data;build a 3D model of the room based upon the measured plurality ofdimensions; receive an indication of a proposed change to the room,wherein the proposed change to the room comprises a remodel of a kitchenor bathroom; modify the 3D model to include the proposed change to theroom; and display a representation of the modified 3D model.
 12. Thecomputer system of claim 11, wherein the proposed change to the roomcomprises addition or removal of a wall.
 13. The computer system ofclaim 11, the system further configured, via the one or more processors,sensors, servers, and/or transceivers, to display a 2D image generatedfrom the 3D model.
 14. The computer system of claim 11, the systemfurther configured, via the one or more processors, sensors, servers,and/or transceivers, to: receive navigation input; and display therepresentation of the modified 3D model by visually navigating throughthe 3D model based upon the received navigation input.
 15. The computersystem of claim 11, the system further configured, via the one or moreprocessors, sensors, servers, and/or transceivers, to: receive cameradata including color data; and build the 3D model by: derivingdimensions of a wall based upon processor analysis of the LIDAR data;deriving a color of the wall based upon processor analysis of the cameradata; and filling, into the 3D model, the wall including the deriveddimensions of the wall and the derived color of the wall.
 16. A computersystem configured to visualize proposed changes to a home, comprising:one or more processors; and a program memory coupled to the one or moreprocessors and storing executable instructions that when executed by theone or more processors cause the computer system to: receive lightdetection and ranging (LIDAR) data generated from a LIDAR camera;measure a plurality of dimensions of a room of the home based uponprocessor analysis of the LIDAR data; build a 3D model of the room basedupon the measured plurality of dimensions; receive an indication of aproposed change to the room, wherein the proposed change to the roomcomprises a remodel of a kitchen or bathroom; modify the 3D model toinclude the proposed change to the room; and display a representation ofthe modified 3D model.
 17. The computer system of claim 16, wherein theproposed change to the room comprises addition or removal of a wall. 18.The computer system of claim 16, wherein the executable instructionsfurther cause the computer system to display a 2D image generated fromthe 3D model.
 19. The computer system of claim 16, wherein theexecutable instructions further cause the computer system to: receivenavigation input; and display the representation of the modified 3Dmodel by visually navigating through the 3D model based upon thereceived navigation input.
 20. The computer system of claim 16, whereinthe executable instructions further cause the computer system to:receive camera data including color data; and build the 3D model by:deriving dimensions of a wall based upon processor analysis of the LIDARdata; deriving a color of the wall based upon processor analysis of thecamera data; and filling, into the 3D model, the wall including thederived dimensions of the wall and the derived color of the wall.